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ABSTRACT 


Faced  with  aging  aircraft  and  fewer  acquisitions.  Naval  Aviation  has  redoubled  its 
effort  to  preserve  assets  through  preventative  maintenance  and  reduction  of  aircraft 
mishaps.  Eighty  percent  of  all  mishaps  are  due  in  part  to  human  error,  and  approximately 
one  out  of  five  major  mishaps  are  due  to  maintained  line,  or  facility  related  factors. 
Among  various  efforts  to  systematically  reduce  mishaps  is  the  use  of  the  Maintenance 
Climate  Assessment  Survey  (MCAS).  This  survey  is  designed  to  capture  maintainer 
perceptions  of  safety.  This  thesis  analyzes  the  results  of  977  responses  to  MCAS  given  to 
the  3rd  Marine  Air  Wing  (MAW)  maintenance  personnel.  In  addition,  it  explores  the 
MCAS’s  relationship  with  human  errors  present  in  21  maintenance-related  incidents 
(MRIs)  using  the  Human  Factors  Analysis  and  Classification  System  -  Maintenance 
Extension.  This  analysis  finds  statistically  different  responses  among  the  squadrons  of 
the  3rd  MAW  to  the  MCAS.  These  differences  show  the  MCAS  can  detect  variations 
between  aviation  units  and  associated  Model  of  Organization  Safety  Effectiveness 
components.  While  no  significant  correlation  between  the  nine  adequately  surveyed 
squadrons  and  their  MRIs  is  found,  a  content  analysis  of  the  MCAS  shows  there  is  a 
relationship  between  the  MRIs  a  squadron  experiences  and  the  items  of  the  six  MOSE 
components. 
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EXECUTIVE  SUMMARY 


Faced  with  aging  aircraft  and  fewer  acquisitions.  Naval  Aviation  has 
redoubled  its  efforts  to  preserve  assets  through  preventative  maintenance  and  reduction  of 
aircraft  mishaps.  While  dramatic  strides  were  achieved  in  the  last  50  years  in  reducing 
the  Class  A  Flight  Mishap  (FM)  rate,  the  presence  of  human  error  has  remained  relatively 
constant.  In  1996,  a  Human  Quality  Management  Board  (HFQMB)  was  established  to 
address  human  errors  in  FMs,  and  initially  targeted  aircrew  error  mishaps.  The  HFQMB 
achieved  positive  results,  but  as  the  last  decade  came  to  a  close  it  was  clear  maintainer 
error  would  have  to  be  targeted  in  order  preserve  aviation  assets. 

The  Naval  Safety  Center  and  the  School  of  Aviation  Safety  at  the  Naval 
Postgraduate  School  are  addressing  aircraft  maintenance  issues  in  order  to  achieve 
additional  Class  A  FM  reductions.  Their  efforts  have  generated  two  tools  to  assess 
maintainer  error  and  trends  and  factors  contributing  to  an  incident,  including  the  safety 
climate  within  which  the  organization  operates.  Schmidt,  Schmorrow  and  Hardee  (1998) 
develop  the  Human  Factors  Analysis  Classification  System  -  Maintenance  Extension 
(HFACS-ME)  taxonomy  used  to  classify  causal  factors  that  contribute  to  maintenance- 
related  incidents  (MRIs).  Schmorrow  (1998),  investigates  all  Naval  Aviation  class  A,  B 
and  C  MRIs  and  Teeters  (1999)  looks  at  all  MRIs  form  the  Fleet  Logistic  Support  Wing. 
Both  use  HFACS-ME  to  analyze  past  mishap  data  building  a  Poisson  model  to  predict 
future  mishaps  and  through  analysis  of  potential  reductions  associated  with  these  error 
types  forecast  a  decline  in  MRIs  resulting  in  significant  cost  savings.  Baker  (1998), 
working  with  a  model  of  safety  effectiveness  based  on  the  concept  of  High  Reliability 
Organizations,  uses  the  existing  Model  of  Safety  Effectiveness  and  the  Command  Safety 
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Assessment  (CSA)  to  develop  a  35-item  Maintenance  Climate  Assessment  Survey 
(MCAS).  Further  work  by  Goodrum  (1999),  Oneto  (1999),  School  of  Aviation  Safety, 
and  AIRPAC  refined  the  MCAS  into  a  43-question  survey. 

This  study  examines  the  MCAS  ability  to  capture  maintainer  perceptions  of 
maintenance  operations  and  explores  the  relationship  between  MCAS  and  aircraft 
maintenance-related  incidents  (MRIs)  using  HFACS-ME.  With  the  study  of  977 
maintainer  surveys  from  the  3rd  MAW,  an  effort  to  assess  the  maintenance  safety  climate 
within  the  wing  is  undertaken.  Commander  3rd  MAW,  realizing  the  School  of  Aviation 
Safety  was  working  on  reducing  MRIs,  decided  to  seek  assistance  to  access  the  reasons 
behind  an  increasing  mishap  rate  in  Fiscal  Year  1999.  The  School  of  Aviation  Safety 
provided  various  training  and  analysis  including:  CSA,  MCAS,  Ground  Crew 
Coordination  Training  (GCT),  Maintenance  Operational  Risk  Management  (MORM) 
awareness  briefs,  the  HFACS-ME  examination  of  Class  A,  B,  C  and  Hazard  Report 
(HAZREP)  MRIs,  and  a  package  on  Class  A  Helicopter  and  tactical  aircraft  (TACAIR) 
mishaps. 

Through  the  use  of  Principal  Component  Analysis,  this  study  validates  the  use  of 
the  MCAS  in  assessing  a  squadron’s  maintenance  safety  climate.  Using  Analysis  of 
Variance  (ANOVA)  and  Multiple  Comparison  testing,  this  study  shows  that  there  is  a 
statistically  significant  difference  between  squadrons.  Tukey’s  Multiple  Comparison  is 
used  to  aggregate  units  into  above  average,  average  and  below  average  maintenance 
safety  climate.  Linear  Regression  is  then  preformed  using  mean  MCAS  responses  as  the 
predictor  variable  and  the  MRI  rate  as  the  response  variable.  While  no  significant 
correlation  between  the  nine  adequately  surveyed  squadrons  and  their  MRIs  is  found,  a 
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content  analysis  of  the  MCAS  shows  there  is  a  correlation  between  the  MRIs  a  squadron 
experiences  and  the  items  of  the  six  MOSE  components. 
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I. 


INTRODUCTION 


“Aviation  in  itself  is  not  inherently  dangerous.  But  to  an  even  greater  degree  than 
the  sea,  it  is  terribly  unforgiving  of  any  carelessness,  incapacity,  or  neglect” 
(Anonymous).  Risk  is  inherent  in  Naval  Aviation,  however  the  success  of  Naval 
Aviation  is  based  on  a  willingness  to  balance  risk  with  the  opportunity  to  take  decisive 
actions  necessary  to  succeed  in  battle  (DON,  1997).  However,  squadron  commanders 
have  a  fundamental  responsibility  to  safeguard  personnel  and  material  resources,  and  to 
accept  only  minimal  risk  to  accomplish  an  assigned  mission  (DON,  1997).  There  have 
been  several  major  accidents  in  which  organizational  error  plays  a  major  role.  One  need 
only  recall  Three  Mile  Island  (March  1979),  the  Space  Shuttle  Challenger  (January 
1986),  and  Chernobyl  (April  1986)  as  examples  of  major  accidents  in  which 
organizational  error  plays  a  significant  role  (Redmill  &  Rajan,  1997).  Roberts  (1990) 
observes  that  some  organizations  operate  in  a  complex  and  hazardous  environment,  yet 
experience  a  very  low  accident  rate.  She  terms  these  activities  “high  reliability 
organizations”  or  “HROs.”  Ciavarelli  (1997)  highlights  research  showing  that  HROs 
have  certain  key  characteristics  in  common  such  as  leadership  style,  management 
policies,  procedures  standardization,  superior  training,  safety  reward,  adequacy  of 
resources  and  staffing,  and  effective  risk  and  hazard  management.  In  a  study  of  nuclear 
powered  aircraft  carriers,  Roberts  (1990)  formally  recognizes  Naval  Aviation  as  an  HRO. 

Despite  Naval  Aviation’s  success  in  reducing  by  50  percent  its  Class  A  (those 
resulting  in  death,  permanent  disability,  or  loss  of  $1  million  or  more)  Flight  Mishap 
(FM)  rate  each  successive  decade  between  Fiscal  Years  (FY)  1950  and  1990,  the 
proportion  of  aircraft  losses  attributable  to  human  error  has  remained  relatively  constant 
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this  decade  (Ciavarelli  &  Figlock,  1997).  Following  the  1996  crash  of  a  Navy  F-14  in 
Nashville,  Tennessee  a  Human  Factors  Quality  Management  Board  (HFQMB)  was 
formed  with  a  goal  to  reduce  Naval  Aviation’s  Class  A  FM  rate  due  to  human  error  by  50 
percent.  The  HFQMB  initially  focused  on  aircrew  error,  which  was  found  to  be  a  casual 
factor  in  60  percent  of  all  mishaps.  However,  it  has  increased  its  focus  to  encompass 
maintenance  error  that  comprises  approximately  15-20  percent  of  Class  A  mishaps. 

A.  BACKGROUND 

The  3rd  Marine  Air  Wing  (MAW)  has  experienced  a  dramatic  rise  in  their  FY 
1999  Class  A  Flight  Mishap  (FM)  rate  and,  intrigued  by  the  efforts  of  the  HFQMB, 
requests  from  the  School  of  Aviation  Safety  at  the  Naval  Postgraduate  school  support  in 
cutting  their  FM  rate.  The  mission  of  the  3rd  MAW  is  to  provide  combat-ready 
expeditionary  aviation  forces  capable  of  short-notice  worldwide  deployment  to  Marine 
Air  Ground  Task  Force  (MAGTF)  fleet  and  unified  commander.  The  main  aircraft  of  3rd 
MAW  are  the  AH-1W  “Super  Cobra”,  UH-1N  “Huey”,  CH-53  “Super  Stallion”,  CH-46E 
“Sea  Knight”,  F/A-18D  “Night  Attack  Hornet”,  F/A-18  “Hornet”,  AV-8B  “Harrier”  and 
the  C-130T  “Hercules”.  The  3rd  MAW  is  composed  of  25  combat  squadrons,  three 
training  squadrons  and  four  Marine  Aviation  Logistics  Squadrons  (MALS)  divided 
among  four  Marine  Air  Groups  (MAGs)  located  on  three  different  Marine  Corps  Air 
Stations  in  Southern  California.  The  MALS  provide  intermediate  maintenance  and 
aviation  logistics  support  to  squadrons  within  each  respective  MAG,  whereas  each 
squadron  performs  it’s  own  organizational  maintenance. 

The  function  of  the  MAGs  is  to  perform  various  related  aviation  tasks  for  the 
Marine  Corps.  The  primary  mission  of  MAG-1 1  (F/A-18,  F/A-18D  &  C-130T)  is  to 
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provide  air  support  to  Marine  Air  Ground  Task  Force  (MAGTF)  commanders.  MAG-13 
(AV-8B)  provides  close-air  support,  conducts  armed  reconnaissance,  and  assumes  limited 
air-defense  roles.  MAG- 16  (CH-53  &  CH-46E)  performs  its  assigned  mission  of 
transportation  and  re-supply  for  Marine  air  and  ground  units.  Finally,  MAG-39  (AH-1W 
&  UH-1N)  provides  utility  helicopter  support,  close-in  fire  support,  fire  support 
coordination,  aerial  reconnaissance,  observation  and  forward  air  control  in  aerial  and 
ground  escort  operations  during  ship-to-shore  movement  and  subsequent  operations 
ashore. 

The  3rd  MAW,  in  dealing  with  is  recent  increased  mishap  rate,  received 
constructive  support  from  the  School  of  Aviation  Safety  at  the  Naval  Postgraduate 
School,  in  the  form  of  training  and  analysis  including:  Command  Safety  Assessment 
(CS  A),  Maintenance  Climate  Assessment  Survey  (MCAS),  Ground  Crew  Coordination 
Training  (GCT),  Maintenance  Operational  Risk  Management  (MORM)  awareness  briefs, 
the  Human  Factors  Analysis  and  Classification  System  -  Maintenance  Extension 
(HFACS-ME)  examination  of  Class  A,  B,  C  and  Hazard  Report  (HAZREP)  maintenance- 
related  incidents  (MRIs),  and  a  package  on  Class  A  Helicopter  and  tactical  aircraft 
(TACAIR)  mishaps.  This  support  was  implemented  to  evaluate  their  maintenance  safety 
climate  in  an  effort  to  understand  the  causes  of  the  ground-based  incidents. 

B.  PURPOSE 

The  intent  of  this  study  is  three  fold:  1)  obtain  3rd  MAW  maintainer  perception  of 
maintenance  operations  safety  with  MCAS,  2)  determine  the  if  human  error  is  a 
significant  factor  in  3rd  MAW  MRIs  using  HFACS-ME,  and  3)  evaluate  the  ability  of  the 
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MCAS  to  detect  differences  in  squadron  maintenance  safety  climate  and  how  it  is 
associated  with  their  MRI  histories. 

C.  PROBLEM  STATEMENT 

During  FYs  1998  and  1999,  3rd  MAW  has  suffered  a  dramatic  increase  in  their 
mishap  rate.  The  commander  3rd  MAW,  recognizing  MRI’s  as  a  part  of  the  problem, 
elected  to  have  the  School  of  Aviation  Safety  use  the  MCAS  to  capture  maintainer 
perceptions  of  3rd  MAW  maintenance  operations  and  HFACS-ME  for  classifying  human 
error  that  contributed  to  the  3rd  MAWs  MRIs.  The  43-question  version  of  the  MCAS  was 
administered  to  the  3rd  Marine  Air  Wing  (MAW)  during  the  last  half  of  1999  in  order  to 
capture  the  maintained  perception  of  their  safety  climate.  In  this  thesis  an  analysis  of  3rd 
MAWs  MRIs  from  January  1998  to  December  1999  is  conducted  to  detect  the  human 
error  present  in  the  incidents  as  identified  by  HFACS  -  ME.  The  results  are  evaluated  in 
an  attempt  to  correlate  these  responses  with  the  MRI  data  over  the  past  24  months.  This 
research  involves  collection  of  survey  results,  generation  of  descriptive  statistics  of  the 
survey,  and  employs  statistical  methods  to  address  the  following  research  questions. 

1.  Are  there  any  noteworthy  findings  with  respect  to  the  MCAS?  Are  there  any 
significant  HFACS-ME  trends  observed  for  the  MRIs? 

2.  Does  the  MCAS  capture  an  overall  attitude  toward  aviation  maintenance 
safety  and  are  all  the  questions  in  each  component  required  to  describe  the 
majority  of  the  variance  of  the  component? 

3.  Does  the  MCAS  detect  maintenance  safety  climate  differences  between 
squadrons,  type  aircraft  and  Marine  Air  Groups  and  are  these  differences 
statistically  significant  as  to  cluster  the  different  squadrons  into  groups? 

4.  Do  squadrons  that  cluster  below  and  above  average  as  related  to  safety  climate 
also  have  a  higher  and  lower  average  number  of  mishaps  respectively? 
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D.  DEFINITIONS 


This  thesis  uses  the  following  definitions  (DON,  1989): 

Naval  Aircraft.  Refers  to  U.S.  Navy,  Naval  Reserve,  U.S.  Marine  Corps,  and  U.S. 

Marine  Corps  Reserve  aircraft. 

Maintenance-Related  Incidents  (MRIs) 

Mishap.  A  Naval  Aviation  mishap  is  an  unforeseen  or  unplanned  event  that 
directly  involves  naval  aircraft,  which  result  in  $10,000  or  greater  cumulative 
damage  to  naval  aircraft  or  personnel.  The  mishap  is  further  divided  into  three 
classes  based  on  the  amount  of  damage  to  the  aircraft,  property  and  personnel 
injury.  The  following  are  the  definitions  of  the  three  classes: 

a.  Class  A.  A  mishap  in  which  the  total  cost  of  property  damage  (including 
all  aircraft  damage)  is  $1,000,000  or  greater;  or  a  naval  aircraft  is 
destroyed  or  missing;  or  any  fatality  or  permanent  total  disability  of  a 
person  occurs  with  direct  involvement  of  naval  aircraft. 

b.  Class  B.  A  mishap  in  which  the  total  cost  of  property  damage  (including  all 
aircraft  damage)  is  $200,000  or  more  but  less  than  $1,000,000  and/or  a 
permanent  partial  disability,  and  or  the  hospitalization  of  five  or  more 
personnel. 

c.  Class  C.  A  mishap  in  which  the  total  cost  of  property  damage  (including  all 
aircraft  damage)  is  $10,000  or  more  but  less  than  $200,000  and/or  injury 
results  in  one  or  more  lost  workdays. 

Hazard  Report  (HAZREP1  An  incident  in  which  the  total  cost  of  property 
damage  (including  all  aircraft  damage)  is  less  the  $10,000  and/or  injury  results  in 
no  lost  workdays. 
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Incidence  rate.  The  total  number  of  Class  A,  B  and  C  mishaps  or  HAZREPS  per  100,000 


flight  hours. 

Aircraft  Ground  Mishap  (AGM).  Those  mishaps  in  which  no  intent  for  flight  existed  at 
the  time  of  the  mishap  and  $10,000  or  greater  aircraft  damage  or  loss  of  an  aircraft, 
and/or  property  damage  occurred. 

Flight  Mishap  (FM).  Those  mishaps  in  which  there  is  $10,000  or  greater  aircraft  damage 
or  loss  of  aircraft  with  the  existence  of  intent  for  flight  for  the  aircraft  existed  at  the 
time  of  the  mishap. 

Flight  Related  Mishap  (FRM).  Those  mishaps  in  which  there  is  no  intent  for  flight  at  the 
time  of  the  mishap  and  $10,000  or  greater  aircraft  damage  or  loss  of  an  aircraft, 
and/or  property  damage  occurs. 

Maintenance  Climate  Assessment  Survey  (MCAS).  A  43 -question  survey  used  to  gain 
insight  into  the  maintenance  community’s  perception  concerning  aviation  mishaps 
within  the  Navy  and  Marine  Corps. 

Human  Factors  Accident  Classification  System  -  Maintenance  Extension  (HFACS-ME). 
A  taxonomic  system  used  to  classify  causal  factors  that  contribute  to  maintenance 
related  mishaps. 

High  Reliability  Organization  (HR.O).  An  organization  that  operates  in  a  hazardous 
environment,  yet  produces  a  very  low  rate  of  accidents  and  incidents  by  operating 
effectively,  safely,  and  having  the  characteristics  of  leadership,  sound  management 
policies,  procedure  standardization,  adequacy  of  resources  and  staffing,  a  defined  system 
for  risk  management,  and  other  factors.  These  traits  can  be  divided  into  six  components: 
1)  Process  Auditing  (PA)  -  an  ongoing  system  to  monitor  hazards;  2)  Reward  System 
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(RS)  -  expected  compensation  or  disciplinary  action  used  to  shape  behavior;  3)  Quality 
Assurance  (QA)  -  policies  and  procedures  for  promoting  quality  performance;  4)  Risk 
Management  (RM)  -  organizational  risk  perception  and  mitigation;  5)  Command  and 
Control  (CC)  -  policies  and  procedures  to  mitigate  risks  and  6)  Communication  / 
Functional  Relationships  (CF)  -  an  environment  where  information  is  freely  exchanged, 
quality  assurance  is  seen  as  a  positive  influence  and  maintenance  workers  are  shielded 
from  external  pressures  to  complete  a  task.  This  sixth  trait  encompasses  a  number  of 
items  specifically  relating  to  maintenance  that  did  not  fit  in  the  original  traits  of  HROs. 

E.  SCOPE  AND  LIMITATIONS 

This  study  uses  the  MCAS  to  survey  the  maintenance  safety  climate  perceptions 
of  the  maintainers  of  the  3rd  MAW  and  to  validate  the  use  of  the  MCAS  as  a  tool  to 
reliably  asses  this  climate.  A  total  of  17  California  based  USMC  Active  Duty  Squadrons 
from  the  3rd  MAW  participate  in  the  survey.  Only  Naval  Aviation  maintenance  personnel 
surveys  are  used  in  this  analysis.  Incomplete  surveys  or  surveys  with  no  variation  in  the 
responses  are  omitted. 

The  survey  is  administered  at  the  various  commands  on  site  and  participants  are 
chosen  from  maintainers  in  the  squadron  in  a  non-random  order.  That  is,  only  those 
maintainers  that  can  participate  in  the  survey  at  the  time  of  administration  are  sent  to 
participate.  The  differences  in  total  surveys  collected  per  squadron  are  due  to  the 
availability  of  maintainers  and  the  commands  willingness  to  participate  in  the  survey. 
Also,  during  the  timeframe  of  this  study,  all  AV-8B  “Harriers”  are  grounded  for  several 
weeks  to  assess  the  reliability  of  certain  components  of  the  airframe. 
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The  Naval  Aviation  Safety  Program  mandates  the  investigation  and  reporting  of 
all  Class  A,  B  and  C  incidents  (OPNAV  3750.6Q,  1989).  However,  there  are  a  number 
of  difficulties  in  using  the  reports  of  these  incidents.  The  information  collected  on 
incidents  is  not  in  HFACS  format  nor  does  it  emphasize  human  factors.  Further,  fidelity 
is  lost  by  transferring  the  incident  investigation  into  the  Safety  Information  Management 
System  (SIMS)  database.  In  addition,  since  OPNAV  3750.6Q  only  recommends  but  does 
not  require  the  reporting  of  HAZEPS  there  number  is  underreported  in  the  SIMS 
database.  Finally,  commands  can  influence  the  recorded  or  reported  cost  associated  with 
a  mishap  thus  lowering  it  from  a  mandatory  reported  Class  C  incident  to  a  non-reported 
HAZREP. 

Chapter  II  contains  a  literature  review  on  investigative  scope,  high  reliability 
organizations,  human  factors  quality  management  board,  maintenance  climate  assessment 
survey  development,  and  human  factors  -  maintenance  extension  development.  In 
Chapter  III  the  methodology  used  in  this  study  is  discussed.  Chapter  IV  presents  the 
results  of  this  study.  Lastly,  Chapter  V  gives  conclusions  include  findings  and 
recommendations. 
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II.  LITERATURE  REVIEW 


An  in  depth  literature  review  is  given  in  this  chapter  to  give  the  reader  an 
introduction  to  the  relationship  between  organizational  effects  and  safety  and  how  two 
tools;  the  Maintenance  Climate  Assessment  Survey  (MCAS)  and  the  Human  Factors 
Analysis  and  Classification  System  (HFACS)  can  be  used  to  understand  this  relationship. 
It  will  begin  with  an  overview  of  high-reliability  organizations.  Next,  a  history  of  the 
Human  Factors  Quality  Management  Board  (HFQMB)  is  discussed  followed  by  MCAS 
and  HFACS-Maintenance  Extension  (ME)  development.  The  chapter  finishes  by 
summarizing  this  body  of  research  and  makes  recommendations  to  tie  HFQMB,  MCAS 
and  HFACS-ME  into  a  three-pronged  strategy  to  analyze  risk  and  recommend  a  course  of 
action  to  prevent  a  mishap  from  occurring  in  aviation  organizations. 

A.  HIGH  RELIABILTITY  ORGANIZATIONS 

One  factor  recognized  as  significantly  contributing  to  a  decrease  in  aircraft 
accidents  is  the  technology  advances  in  the  design  and  maintenance  of  modem  military 
aircraft  (Roberts,  1990).  However,  Roberts  (1990)  finds  this  level  of  technical 
complexity  makes  military  aircraft  vulnerable  to  accidents  or  failures  that  cannot  be 
predicted  during  the  design  process.  In  an  effort  to  understand  why  certain  organizations 
having  a  high  level  of  complexity  operating  in  a  hazardous  environment  do  not 
experience  greater  accidents  as  complexity  increases,  she  observes  that  these 
organizations  have  certain  factors  in  common.  She  calls  these  organizations  “High 
Reliability  Organizations”  or  “HROs”. 

Technology  in  general,  aids  in  reducing  mishaps,  but  the  existing  organizational 
research  is  inadequate  to  understand  what  organizational  processes  define  HROs 
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(Roberts,  1990).  Her  research  shows,  that  HROs  have  like  leadership  style,  management 
policies,  procedures  standardization,  superior  training,  and  reward  systems.  These  ideas 
and  research  by  Libuser  (1994)  lead  to  the  development  of  a  Model  of  Organizational 
Safety  Effectiveness  (MOSE).  The  MOSE  model  organized  HRO  traits  into  the  five 
categories  of  Process  Auditing,  Reward  System,  Quality  Assurance,  Risk  Management 
and  Command  and  Control  (Libuser,  1994).  The  five  MOSE  components  are 
summarized  as  follows: 

1.  Process  Auditing  -  a  system  of  ongoing  checks  to  identify  a  hazardous 
condition  and  a  process  to  take  corrective  actions. 

2.  Reward  System  -  a  system  to  recognize  and  reward  safe  behaviors,  and  to 
discourage  unsafe  behaviors. 

3.  Quality  Assurance  -  Sets  control  procedures  to  monitor  quality  and  correct 
deviations  to  referenced  standards. 

4.  Risk  Management  -  Systematic  process  to  identify  and  manage  risk. 

5.  Command  &  Control  -  an  environment  where  information  is  freely 
exchanged,  quality  assurance  is  seen  as  a  positive  influence  and  maintenance 
workers  are  shielded  from  external  pressures  to  complete  a  task. 

B.  HUMAN  FACTORS  QUALITY  MANAGEMENT  BOARD 

In  an  attempt  to  address  human  error  in  Naval  Aviation  mishaps,  a  HFQMB  was 
established  in  1996.  Its  charter  is  to  analyze  human  factor  involvement  in  Naval  Aviation 
mishaps  and  in  Naval  Aviation  operations  and  develop  interventions  for  them.  Initially, 
the  HFQMBs  goal  was  to  cut  the  mishap  rate  associated  with  human  error  in  half 
(Nutwell  &  Sherman,  1997).  The  HFQMB  used  a  “three-prong  approach”:  1)  Mishap 
Data  Analysis,  2)  Benchmarking  and  3)  Climate  Safety  Assessment  to  identify  human 


10 


error,  uncover  the  best  practice  and  find  process  improvements,  and  assess  safety  climate, 
respectively  (See  Figure  1). 


Mishap  Data 
Analysis 


Benchmarking 


Command  Safety 
Assessment 


Figure  1.  HFQMB  Methodology  (Schmidt,  1998) 


Faced  with  aircrew  error  contributing  to  three  out  of  five  (Naval  Safety  Center,  " 
1996)  Class  A  Flight  Mishaps  (FMs)  the  HFQMB  focused  initially  on  them,  and  it 
consequently  resulted  in  significant  reductions  in  the  Class  A  FM  rate.  These  results 
coupled  with  emerging  requirements  to  concentrate  on  aging  aircraft  related  issues 
convinced  the  HFQMB  to  expand  its  scope  to  include  human  factors  in  maintenance- 
related  mishaps  (MRMs)  using  the  same  process  (Schmidt,  Schmorrow  &  Hardee,  1998). 

C.  MCAS  DEVELOPMENT 

Ciavarelli  and  Figlock  (1997)  develop  a  67-item  prototype  “Command  Safety 
Assessment”  (CSA)  .for  aircrew  based  on  the  descriptive  MOSE  components  derived 
primarily  from  the  works  of  Libuser  (1994),  Roberts  (1990),  Zohar  (1980).  Through  the 
survey  of  1,254  naval  aviators  and  flight  officers  they  are  able  to  differentiate  between 
squadrons  having  high  scores  for  HRO  components  and  those  with  low  ones.  This 
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research  represents  a  first  attempt  to  focus  on  key  organizational  issues  in  order  to 
improve  our  understanding  of  the  possible  influence  that  a  command  may  have  in  the 
chain  of  events  leading  to  a  maintenance-related  aircraft  mishap. 

Where  Ciavarelli  and  Figlock  focus  on  organizational  effects  relating  to  aircrew 
safety  issues,  Baker  (1998),  Goodrum  (1999),  and  Oneto  (1999)  focus  on  the 
organizational  effects  pertaining  to  maintainer  safety  issues.  Baker  (1998)  uses  the 
existing  CSA  and  MOSE  to  develop  a  prototype  MCAS.  It  consists  of  15  demographic 
and  67  maintenance-related  items.  The  demographic  items  include  inquiries  about  rank, 
community,  shift  worked,  total  years  of  service,  and  total  years  of  aviation  maintenance 
experience.  The  maintenance-related  items  come  from  a  candidate  battery  of  over  200 
items  developed  from  the  CSA,  a  Naval  Safety  Center  (NAVSAFCEN)  questionnaire, 
and  noted  questions  from  subject  matter  experts.  Maintenance  experts  review  this  initial 
battery  of  questions  and  sort  them  according  to  the  MOSE  components.  A  number  of 
items  not  fitting  in  the  original  five  categories  are  placed  in  a  sixth  called 
Communication/Functional  Relationship  (CF)  -  an  environment  where  information  is 
freely  exchanged,  quality  assurance  is  seen  as  a  positive  influence  and  maintenance 
workers  are  shielded  from  external  pressures  to  complete  a  task.  The  questions  are  then 
organized  according  to  their  MOSE  components. 

Bakers’  (1998)  study  includes  268  participants  from  three  reserve  squadrons  that 
represented  a  spectrum  of  aviation  communities.  He  tests  the  reliability  and  consistency 
of  the  prototype  survey  with  a  squadron  from  the  Fleet  Logistic  Support  (VR)  Wing. 
With  the  survey  shown  to  be  content  valid.  Baker  (1998)  administers  the  prototype 
MCAS  to  two  other  groups  of  squadron  maintenance  personnel  from  the  Maritime  Patrol 
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and  Reserve  Helicopter  Wing  squadrons  located  at  Moffett  FAF  and  NAS  North  Island, 
respectively.  Studying  these  Naval  Air  Reserve  Force  squadrons  as  well  as  input  from 
the  Naval  Safety  Center  and  the  School  of  Aviation  Safety  his  research  results  in  a  35 
item,  six-component  survey  using  a  Likert  type  rating  scale  to  assess  safety  in 
maintenance  operations. 

Goodrum  (1999)  works  with  VR  Wing  and  surveys  nearly  1000  maintainers  from 
three  different  aircraft  communities  (C-9B,  C-20,  &  C-130T).  The  results  of  the  study 
concludes  that  the  MCAS  developed  by  Baker  (1998)  can  be  utilized  as  a  tool  for 
effectively  capturing  an  aviation  maintainer’s  perceptions  of  safety  in  maintenance 
operations.  Through  analysis  of  the  MCAS  responses  a  number  of  different  areas  related 
to  the  MOSE  components  are  identified  as  raising  concern  within  the  different 
communities. 

Oneto  (1999),  studying  assorted  aircraft  communities  within  the  Naval  Air 
Reserve  Force  (NARF),  also  builds  upon  Bakers  (1998)  research,  surveying  439 
maintenance  personnel  of  the  NARF.  Oneto ’s  working  independently  from  Goodrum 
also  concludes  that  the  MCAS  is  an  effective  tool  for  capturing  an  aviation  maintainer’s 
perceptions  of  safety  in  maintenance  operations.  However,  in  his  study  of  NARF 
squadrons,  Oneto  fails  to  identify  any  single  MOSE  component  responsible  for 
controlling  the  outcome  of  the  survey. 

Goodrum  and  Oneto’ s  combined  studies:  1)  validate  the  stability  of  the  survey 
using  Cluster  Analysis  by  achieving  similar  conclusions  through  the  use  of  two 
diametrically  functioning  algorithms  and  2)  validate  the  individual  survey  questions  using 
principal  component  analysis.  Further,  both  studies  conclude  by  proposing  a  nearly 
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identical  40-item  MCAS.  Armed  with  this  new  revision,  the  Naval  Safety  Center,  Naval 
Air  Forces  Pacific  and  School  of  Aviation  Safety  implemented  revisions  to  come  up  with 
the  current  43-item  MCAS. 

D.  HFACS-ME  DEVELOPMENT 

The  HFACS  is  a  tool  to  analyze  human  errors  contributing  to  Naval  Aviation 
mishaps  that  was  developed  by  Naval  Aerospace  Psychologists  (Wiegman  &  Shappell, 
1997)  of  the  Naval  Safety  Center.  (Schmidt  et  al,  1998).  HFACS  incorporates  features  of 
Bird’s  ‘Domino  Theory,’  Edward’s  ‘SHEL  Model,’  and  Reason’s  ‘Swiss  Cheese  Model.’ 
(Schmidt,  et  al,  1998).  Latent  conditions  and  active  failures  are  partitioned  into  one  of 
three  top-level  categories.  Unsafe  Supervision  and  Operator  Conditions  are  latent 
conditions  while  unsafe  Acts  of  the  Operator  are  active  failures.  Latent  conditions  are  the 
result  of  decisions  made  by  individuals  who  are  not  in  direct  control  of  the  system 
(Zotov,  1996),  and  Active  failures  are  those  typically  produced  by  “front-line”  operators 
of  the  system  (Reasons,  1998). 

Drawing  from  HFACS  a  maintenance  extension  (ME)  taxonomy  was  developed 
to  classify  causal  factors  that  contribute  to  Maintenance  Related  Mishaps  (MRMs).  This 
addition  to  HFACS  consists  of  four  broad  human  error  categories  (see  Figure  2): 
Supervisory  Conditions  (latent),  Working  Conditions  (latent),  Maintainer  Conditions 
(latent),  and  Maintainer  Acts  (active).  Under  the  umbrella  of  organizational  climate  are 
the  latent  factors,  Supervisory,  Maintainer,  and  Working  Conditions  that  can  impact  a 
maintainer’s  performance  and  can  contribute  to  an  active  failure,  an  unsafe  maintainer 
act.  These  acts  may  directly  lead  to  an  incident  (mishap  or  injury).  The  unsafe 
maintainer  act  may  also  become  a  latent  maintenance  condition,  which  the  aircrew  may 
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encounter  while  operating  the  aircraft.  In  addition,  latent  supervisory  conditions  may 
also  lead  directly  to  maintenance  conditions. 

Examples  of  how  this  taxonomy  is  applied  for  the  first  and  second  order  error 
types  are  now  given.  Latent  Supervisory  Conditions  that  can  contribute  to  an  active 
failure  include  both  unforeseen  conditions  and  those  conditions  caused  by  the  squadron 
supervisory  environment.  An  example  of  unforeseen  supervisory  condition  is  an  engine 
that  falls  off  of  a  stand  due  to  an  unforeseen  hazard  of  high  seas  state.  An  example  of  a 
squadron  supervisory  condition  is  a  supervisor  who  does  not  ensure  that  maintenance 
personnel  are  wearing  required  personal  protective  gear. 


Figure  2.  HFACS  Maintenace  Extension  Diagram 


Latent  Maintainer  Conditions  that  can  contribute  to  an  active  failure  include 
medical,  crew  coordination,  and  readiness.  An  example  of  a  maintainer  medical 
condition  is  a  maintainer  who  has  a  marital  problem  and  cannot  focus  on  a  maintenance 
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action.  An  example  of  maintainer  crew  coordination  is  a  maintainer  who  performs  a  task, 
counter  to  standard  procedures,  because  the  maintainer  is  overly  submissive  to  a  superior. 
An  example  of  maintainer  readiness  conditions  is  a  maintainer  who  is  working  on  an 
aircraft  skipped  the  requisite  on  the  job  training  evolution. 

Latent  Working  Conditions  that  contribute  to  an  active  failure  include 
environmental,  equipment  and  workspace.  An  example  of  environmental  working 
conditions  is  a  maintainer  who  is  working  at  night  on  the  flightline  does  not  see  a  tool  he 
left  behind.  And  example  of  equipment  working  conditions  is  a  maintainer  who  is  using 
a  defective  test  set  does  not  precheck  it  before  troubleshooting.  An  example  of 
workspace  working  conditions  is  a  maintainer  who  is  working  in  a  hangar  bay  cannot 
properly  position  the  maintenance  stand. 

Maintainer  Acts  are  active  failures,  which  directly  or  indirectly  cause  accidents, 
or  lead  to  Latent  Maintenance  Condition,  the  include  errors  and  violations.  An  example 
of  errors  in  maintainer  acts  is  a  maintainer  who  is  very  familiar  with  a  procedure  might 
reverse  steps  in  a  sequence.  An  example  of  violations  in  maintainer  acts  is  a  maintainer 
who  engages  in  practices,  condoned  by  management,  that  bend  the  rules. 
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Table  1  shows  the  complete  HFACS-ME  categories  having  three  orders:  first,  second 
and  third  that  reflect  a  decomposition  of  the  error  types  from  macro  to  micro  (Schmidt  et 
al,  1998). 


First  Order 

Second  Order 

Third  Order 

Supervisory 

Conditions 

Unforeseen 

Hazardous  Operations 
Inadequate  Documentation 
Inadequate  Design 

Squadron 

Inadequate  Supervision 
Inappropriate  Operations 
Failed  to  Correct  Problem 
Supervisory  Violations 

Maintainer 

Conditions 

Medical 

Mental  State 

Physical  State 
Physical/Mental  Limitation 

Crew  Coordination 

Communication 

Assertiveness 

Adaptability/Flexibility 

Readiness 

Preparation/Training 

Qualification/Certification 

Violation 

Working 

Conditions 

Environment 

Lighting/Light 

Exposure/W  eather 
Environmental  Hazards 

Equipment 

Damaged 

Unavailable 

Dated/Uncertified 

Workspace 

Confining 

Obstructed 

Inaccessible 

Maintainer 

Acts 

Error 

Attention 

Memory 

Rule/Knowledge 

Violation 

Routine 

Infraction 

Exceptional 

Table  1.  HFACS  Maintenance  Extension  Categories 


Organizational  climate  can  effect  the  three  types  of  universal  human  activity  that 
are  common  in  hazardous  environments:  control  under  normal  conditions,  control  under 
emergency  conditions  and  maintenance-related  activities.  Reason  (1997)  defines 
maintenance-related  to  include  unscheduled  repairs,  inspections,  planned  preventive 
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maintenance  and  calibration  and  testing.  He  further  analyzes  these  activities  and 
contends  that  it  would  be  reasonable  to  expect  that  an  activity  scoring  “high”  in  all  three 
areas  is  the  most  likely  to  have  a  human  performance  problem.  Of  those  universal  human 
activities,  Reason  (1997)  shows  that  these  maintenance-related  errors  pose  the  largest 
human  factors  problem  (see  Table  2). 


Activity 

‘Hands  on’ 

Criticality 

Frequency 

Normal  control 

Low 

Moderate 

High 

Emergency  control 

Moderate 

High 

Low 

Maintenance-related 

High 

High 

High 

Table  2.  The  Relative  Likelihood  of  Human  Performance 
Problems  in  the  Universal  Human  Activities 


With  this  analysis  complete  Reason  (1997)  looks  at  which  aspect  of  maintenance 
is  most  likely  to  be  associated  with  “less-than-adequate”  human  performance.  He 
concludes  “that  regardless  of  the  domain,  all  maintenance-related  activities  require  the 
removal  of  fastenings  and  the  disassembly  of  components,  followed  by  their  reassembly 
and  installation”  the  latter  of  which  he  identifies  as  the  biggest  problem  (see  Figure  3). 


Figure  3,  The  bolt-and-nuts  example 


Reason  (1999)  argues  that  since  there  is  only  one  way  to  remove  the  nuts,  the  chances  of 
making  an  error  are  greatly  reduced.  However,  in  the  case  of  installation  there  are  over 
40,000  ways  in  which  the  nuts  can  be  reassembled  in  the  wrong  order.  While  this 
example  may  seem  simple,  it  is  also  supported  by  available  evidence  from  leading 
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aircraft  manufactures  that  report  that  installation  errors  comprise  over  70  percent  of  the 
contributing  factors  to  maintenance-related  events. 

Reason  (1997)  further  explains,  errors  committed  by  pilots,  control  room 
operators  and  other  “sharp-end”  personnel  may  be  the  last  straw  leading  to  an 
organizational  mishap.  However,  it  is  the  latent  conditions  created  by  maintenance- 
related  mistakes  that  create  the  proper  environment  to  start  an  accident  in  motion  or  make 
it  impossible  to  recover  when  the  error  presents  itself  (Reason,  1997).  By  the  same 
token,  even  though  Reason  (1997)  paints  a  rather  bleak  picture  with  regard  to 
maintenance-related  errors,  he  states  that  they  also  offer  the  greatest  opportunity  for 
human  factors  improvements. 

To  identify  aviation  safety  concerns  related  to  maintenance,  Schmorrow  (1998) 
examines  Naval  Aviator  maintenance  mishaps  using  HFACS-ME.  His  research,  based 
on  a  prior  study  by  Schmidt,  Schmorrow  and  Hardee  (1998),  develops  a  method  to 
quantitatively  determine  the  existence  of  significant  patterns  of  human  error  involved  in 
flight  mishaps.  Using  HFACS-ME,  Teeters  (1999)  considers  MRMs,  Hazardous  Reports 
(HAZREPS)  and  Personal  Injury  Reports  (PIPs).  He  shows  that  if  reductions  in  causal 
factors  can  be  achieved,  then  the  numbers  of  incidents  related  to  these  causal  factors 
should  be  reduced. 

E.  SUMMARY 

In  the  face  of  diminishing  budgets  and  reduced  manning,  during  the  past  decade 
Naval  Aviation  has  been  faced  with  the  problem  of  maintaining  operational  readiness  in  a 
“make  do”  environment  (Schmorrow,  1998).  HROs  such  as  Naval  Aviation  with  their 
ever-increasing  reliance  on  advanced  and  complex  systems  need  ways  to  identify  and 
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reduce  human  errors.  The  HFQMB  with  its  initial  focus  on  aircrew  error  has  expanded 
the  scope  of  effort  to  include  maintenance-related  errors.  Two  tools  used  in  this  effort, 
are  the  MCAS  and  HFACS-ME.  Developed  through  coordinated  efforts  between  Air 
Forces  Pacific,  Naval  Safety  Center,  and  the  School  of  Aviation  Safety  at  the  Naval 
Postgraduate  School,  it  has  been  shown  that  these  tools  are  effected  in  1)  effectively 
measuring  and  organization’s  safety  climate  and  2)  classifying  the  errors  present  in  those 
climates.  It  is  hypothesized,  that  through  the  use  of  these  devices  one  could  assess  a  units 
risk  and  through  analysis  of  the  data  recommend  a  preventive  course  of  action  to  prevent 
a  mishap  from  occurring  Naval  Aviation  organizations. 
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III.  METHODOLOGY 


A.  RESEARCH  APPROACH 


This  study  involves  the  analysis  of  data  obtained  from  a  43  item  Maintenance 
Climate  Assessment  Survey  (MCAS),  given  to  maintenance  personal  of  the  3rd  MAW. 
This  survey  is  based  on  a  model  of  HROs.  The  survey  results  are  analyzed  to  determine 
the  surveys  validity  and  reliability.  In  addition,  the  survey  results  are  then  used  to 
determine  if  they  can  be  associated  with  observed  maintenance-related  incidents  (MRIs). 


MRIs  are  classified  in  accordance  with  the  Human  Factors  Analysis  and  Classification 


System  -  Maintenance  Extension  (HFACS-ME)  taxonomy  to  observe  human  error 


frequency  and  patterns. 

B.  DATA  COLLECTION 


1.  Subjects 


A  total  of  976  officer  and  enlisted  personnel  involved  in  aviation  maintenance  in 
the  3rd  Marine  Air  Wing  (MAW)  participate  in  this  study.  The  3rd  MAW  units  are 
located  at  Marine  Corps  Air  Station  Miramar,  California,  Camp  Pendleton,  California, 
and  Marine  Corps  Air  Station  Yuma,  Arizona  (See  Figure  4  for  the  3rd  MAW  command 


structure). 


—  MAG  1 1  (Miramar) 

- MALS  11 

- VMFA  121 

- VMFA  225 

- VMFA  242 

- VMFAT  101 

- VMFA  232 

- VMFA  314 

— VMFA  323 
- VMGR  352 


-  MAG  13  (Yuma) 

— MALS  13 

- VMA211 

- VMA214 

— VMA311 
- VMA513 


—  MAG  16  (Miramar) 

- MALS  16 

- HMH  465 

- HMH  462 

- HMH  361 

- HMM466 

- HMM  161 

— HMM  163 

- HMM  165 

- HMM  166 


—  MAG  39  (Pendleton) 

— MALS  39 
— HMLA  169 
— HMLA  267 
— HMT  303 
— HMLA  367 
— HMLA  369 
— HMMT164 
— HMM  268 
- HMM  364 


Figure  4.  3rd  MAW  Command  Structure 
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Four  mishaps  and  five  hazard  reports  (HAZREPs)  data  from  the  3rd  MAW 
collected  by  the  Naval  Safety  Center  are  used.  The  accidents  in  these  reports  occur 
between  January  1998  to  December  1999. 

2.  Instruments 

a.  MCAS 

The  MCAS  a  self-administered,  group  survey  consisting  of  six 
demographic  and  43  maintenance  related  items.  The  maintenance  items  of  the  MCAS  are 
grouped  into  six  categories  or  components:  (process  auditing,  reward  system,  quality,  risk 
management,  command  and  control,  and  communication/functional  relationships).  The 
survey  uses  a  five  point  Likert  rating  scale  with  verbal  anchors  as  follows:  Strongly 
Disagree,  Disagree,  Neutral,  Agree,  and  Strongly  Agree. 

b.  HFACS-ME 

HFACS-ME  is  used  to  classify  casual  factors  that  contribute  to  MRMs. 
This  device  uses  a  taxonomy  that  consists  of  four  broad  human  error  categories: 
Supervisory  Conditions  (latent),  Working  Conditions  (latent),  Maintainer  Conditions 
(latent),  and  Maintainer  Acts  (active).  This  “first  order  effect”  is  further  divided  into 
second  order  and  third  order  effects  that  decompose  the  error  types  from  a  macro  to 
micro  perspective.  Since  only  first  and  second  order  casual  factors  Table  2  shows  the 
reduced  HFACS-ME  categories.  The  mishap  and  HAZREP  data  is  classified  into  first 
order  causal  factors  that  contribute  to  the  MRMs  using  the  Human  Factors  Accident 
Classification  System  -  Maintenance  Extension  (HFACS-ME). 
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First  Order 

Second  Order 

Supervisory 

Conditions 

Unforeseen 

Squadron 

Maintainer 

Conditions 

Medical 

Crew  Coordination 

Readiness 

Working 

Conditions 

Environment 

Equipment 

Workspace 

Maintainer 

Acts 

|  Error 

Violation 

Table  3.  HFACS  ME  1st  &  2nd  Order  Categories 
3.  Procedure 
a.  MCAS 

The  survey  is  administered  on  site  and  in  a  group  setting  at  the  various 
participating  squadrons  of  the  3rd  MAW.  Additionally,  the  survey  is  given  in  conjunction 
with  a  scheduled  maintenance  safety  presentation  on  human  factors  issues  in  aviation. 
The  squadrons  are  in  various  stages  of  training  and  operational  tasking  at  the  time  of  the 
survey.  Differences  in  operational  tasking  account  for  much  of  the  variability  in  the 
number  of  surveys  collected  from  each  squadron.  For  example,  a  squadron  not  in  the  area 
due  to  deployment  during  the  time  of  the  survey  would  not  have  been  included  in  this 
study.  The  potential  MCAS  respondents  are  briefed  on  the  survey  and  its  purpose. 
Questions  that  arise  pertaining  to  the  survey  are  answered  by  the  survey  administer.  All 
surveys  are  collected  immediately  upon  completion  to  allow  for  maximum 
accountability. 
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b.  Mishap /HAZREP  Data 

The  Naval  Safety  Center  (NAVSAFCEN)  database  was  queried  for  all 


Marine  Forces  Pacific  (MARFORPAC)  incidents  between  Fiscal  Year  (FY)  1997  to 
February  2000. 

C.  DATA  ANALYSIS 

1.  Data  Tabulation 

a.  MCAS 

Survey  data  sheets  are  scanned  into  a  text  file  using  a  Scantron  8200  with 
Scanform  version  3.21  software  on  an  IMB  compatible  computer  running  Windows  3.1. 
The  text  file  is  converted  into  Microsoft  Excel  2000  format.  The  spreadsheet  consists  of 
976  rows  of  respondents  and  51  columns  containing  demographic  variables  and  survey 
responses.  The  first  eight  columns  contain  the  demographic  variables  and  the  remaining 
43  columns  hold  the  subjects  responses  to  questions  1  thru  43  of  the  survey.  The 
demographic  variables  are:  1)  aircraft  community,  2)  squadron  number,  3)  rank,  4)  total 
years  of  aviation  maintenance  experience,  5)  work  center  and  6)  shift.  All  demographic 
information  is  then  replaced  by  three  variables  indicating  the  squadron  number,  type 
aircraft  and  marine  air  group  (MAG)  from  which  the  survey  results  are  obtained.  The  43 
columns  are  further  partitioned  into  six  sets  or  components.  The  six  components  are:  1) 
Process  Auditing  (questions  1-6),  2)  Reward  Systems  and  Safety  Climate  (questions  7- 
14),  3)  Quality  Assurance  (questions  15-20),  4)  Risk  Management  (questions  21-29),  5) 
Command  and  Control  (questions  30-37),  and  6)  Communications  (questions  38-43). 
The  responses  in  each  column  are  scored  as  1,2, 3, 4  or  5  corresponding  to  the  Likert  scale 
Strongly  Disagree  (E),  Disagree  (D),  Neutral  (C),  Agree  (B),  Strongly  Agree  (A), 
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respectively.  Question  21  is  negative  in  wording  compared  to  all  other  questions  of  the 
survey.  Survey  questionnaire  items  that  have  no  response  are  left  blank.  Questionnaires 
that  have  blank  entries  for  any  question  or  item,  or  for  which  the  same  answer  is  given  for 
all  questions  are  eliminated.  A  complete  list  of  removed  data  is  available  in  the  results 
section. 

b.  HFACS-ME 

A  request  is  made  to  Naval  Safety  Center  to  query  the  database  in  order  to 
provide  all  maintenance-related  mishap  and  HAZREP  data  for  all  Marine  Forces  Pacific 
commands  between  the  years  1996  to  February  2000.  This  query  generates  a  text  file 
from  which  is  culled  all  mishap  and  HAZREP  data  for  commands  of  the  3rd  MAW  from 
October  1997  to  February  2000.  These  are  then  divided  into  Class  A,  B,  C  and  HAZREP 
categories  and  are  then  evaluated  into  the  HFACS-ME  first  and  second  order  casual 
factors  by  the  Naval  Postgraduate  School  for  Aviation  Safety.  Finally,  simple  counts  are 
generated  for  the  first  and  second  order  causal  factors  and  these  factors  are  then  weighted 
to  give  an  overall  accident  value  for  each  squadron,  type  and  MAG.  These  values  are 
used  to  create  a  3rd  MAW  maintenance-related  error  table. 

2.  Statistical  Analysis 

a.  MCAS 

Summary  statistics  are  computed  for  each  squadron,  type  aircraft  and 
MAG  in  order  to  describe  basic  and  general  information  about  the  question  results.  In 
particular,  mean  response  for  each  question  is  computer  by  squadron,  type  aircraft  and 
MAG.  In  addition,  the  questions  within  each  of  the  six  MOSE  components  are  averaged 
for  each  respondent.  These  averages  are  then  used  as  the  independent  variable  in  an 
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Analysis  of  Variance  (ANOVA)  to  see  if  there  is  a  difference  in  expected  responses  for 
different  squadrons  and  for  different  MOSE  components.  It  is  noted  that  non-parametric 
analysis  could  also  be  done  by  replacing  Likert  scores  with  ranks  before  doing  the 
ANOVA.  However  such  non-parametric  methods  are  a  form  of  scoring  and  all  methods 
are  considered  problematic  (Kimeldorf,  Sampson  &  Whitaker,  1992).  Therefore,  the 
very  errors  that  a  non-parametric  approach  might  attempt  to  correct  might  in  and  of 
themselves  introduce  new  unforeseen  errors  along  with  making  the  resulting  analysis 
difficult  to  interpret  to  the  population  under  study.  Additionally,  standard  deviations  are 
tabulated  for  the  MOSE  components  and  individual  questions  for  each  squadron,  type 
aircraft  and  MAG.  The  S-Plus  2000  (Mathsoft,  1999)  program  is  used  for  multivariate 
statistical  analysis.  Principal  Component  methodologies  are  used  to  validate  the  stability 
of  the  MCAS  with  a  new  population. 

b.  HFACS-ME 

Summary  statistics  are  developed  for  each  squadron  in  order  to  describe 
basic  and  general  information  about  the  maintenance-related  error.  Simple  linear 
regression  is  conducted  with  mean  MCAS  score  as  the  independent  variable  and 
weighted  MRI  as  the  dependent  variable.  In  addition,  once  complete  an  ad  hoc  procedure 
is  conducted  to  see  if  the  perceived  safety  climate  predicted  by  the  MCAS  is  related  to  an 
appropriate  level  of  maintenance-related  errors  for  those  squadrons. 
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IV.  RESULTS 


A.  MCAS  DESCRIPTIVE  STATISTICS 

1.  Sample 

Maintenance  Climate  Assessment  Surveys  (MCAS)  are  collected  from  977 
military  personnel  serving  in  squadrons  of  the  3rd  Marine  Air  Wing  (MAW).  Of  these, 
235  were  in  Marine  Aviation  Logistics  Squadrons  (MALS)  and  these  surveys  are  not 
included  in  the  study  because  these  squadrons  provide  intermediate  maintenance  and  do 
not  have  aircraft  or  aircrew  directly  assigned  to  them.  Additionally,  61  surveys  are 
removed  from  this  remaining  group  either  because  of  missing  values  (55)  or  the  same 
response  to  all  43  questions  (six).  The  remaining  681  surveys  will  be  addressed  in  the 
MCAS  Results  section  of  this  thesis. 

2.  Analysis  of  Removed  Surveys 

An  effort  is  taken  to  determine  if  removing  the  61  surveys  with  missing  values  or 
constant  response  introduces  bias  in  the  Principal  Component  Analysis.  Of  the  55 
surveys  removed  due  to  missing  values,  there  are  a  total  of  74  questions  left  blank.  Table 
4  shows  the  questions  associated  with  a  missing  value  and  the  number  of  times  that  the 
particular  question  is  not  answered.  Of  note,  are  questions  17,  29,  36,  42  and  43,  they 
are:  17)  “QARs/CDIs  sign-off  after  required  actions  are  complete  and  are  not  pressured 
by  supervisors  to  sign-off,”  29)  “Based  upon  my  command’s  current  assets/manning  it  is 
not  over-committed,”  36)  “All  maintenance  evolutions  are  properly  briefed,  supervised, 
and  staffed  by  qualified  personnel,”  42)  “Maintenance  Control  troubleshoots/resolves 
gripes  before  flight,”  and  43)  “Maintainers  are  briefed  on  potential  hazards  associated 
with  maintenance  activities.  These  questions  account  for  approximately  64  percent  of  the 
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missing  values,  with  question  43  accounting  for  nearly  33  percent.  In  addition,  1 1  out  of 
the  12  people  who  fail  to  answer  question  42  also  fail  to  answer  question  43.  According 
to  Reasons  (1998)  this  effect  is  observed  for  questions  at  the  end  of  surveys  producing 
what  he  calls  “end  effect.”  Because  questions  42,  and  43  are  the  only  questions  on  the 
last  page  of  the  survey  this  effect  is  amplified. 


# 

Questions 

3,5,9,10,11,14,15,16,18,20,24,26,32,33,37,40 

1 

1,2,4,6,8,12,13,19,21,22,27,28,30,31,34,35,39 

2 

7,23,25,38,41 

3 

17 

4 

29,36 

12 

42 

24 

43 

Table  4.  Number  Response  Missing  by  Question 


To  see  if  the  61  surveys  differ  from  the  other  681  surveys,  the  demographic  variables 
of  rank  and  squadron  are  compared.  Table  5  lists  the  frequency  and  percentages  of  the 
surveys  removed  by  rank.  In  Figure  5,  these  are  compared  to  the  surveys  retained  in  the 
analysis.  This  figure  shows  that  those  surveys  removed  are  representative  of  the 
remaining  surveys  used  for  the  Principal  Component  Analysis  and  no  bias  should  be 
introduced  by  their  removal. 


Blank 

El-3 

E4-5 

E6-9 

Officer 

# 

% 

# 

% 

# 

% 

# 

% 

# 

% 

1 

1.7% 

32 

53.3% 

21 

6 

10.0% 

0 

0.0% 

Table  5.  Percentage  and  Count  of  Removed  Surveys  by 
Participant  Rank  Response 
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Table  6  shows  the  frequency  of  surveys  removed  from  a  particular  squadron.  The 
tables  shows  a  range  of  removed  surveys  between  two  percent  for  squadron  B  and  17 
percent  for  squadron  F  with  an  overall  average  of  10  percent  surveys  that  have  either  a 
missing  value  or  the  same  response  for  all  questions. 


Figure  5.  Comparison  of  Retained  and  Removed  Surveys  by  Rank 


A 

B 

C 

E 

F 

G 

H 

I 

J 

K 

L 

M 

Q 

13% 

17% 

12% 

11% 

11% 

11% 

9% 

16% 

7% 

Table  6.  Frequency  by  Squadrons  of  Surveys  Removed 


3.  Demographics 

The  number  of  survey  participants  are  given  by  rank  and  squadron  in  Table  7.  In 
this  table,  participant  squadrons  and  Marine  Air  Groups  (MAGs)  are  randomly  assigned 
letters  in  place  of  the  squadron  identifier.  Approximately  84.9  percent  of  those 
participating  in  the  MCAS  are  enlisted  maintained  serving  in  the  ranks  of  El  -  E5.  This 
percentage  agrees  with  the  79.2%  observed  by  Goodrum  (1999)  in  his  study  of  squadrons 
of  the  Fleet  Logistics  Support  Wing  and  the  81%  observed  by  Oneto  (1999)  in  his  study 
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of  aircraft  communities  within  the  Naval  Air  Reserve  Force.  Similar  proportions  for  all 
squadrons  are  observed  for  the  MCAS  participants  except  those  units  that  contribute  a 
low  number  of  surveys  as  shown  in  Figure  6.  Histograms  of  other  descriptive 
demographic  variables  such  as:  1)  Total  Years  Maintenance  Experience,  2)  Work  Center, 


and  3)  Shift  are  given  in  Appendix  (C). 
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Table  7.  Number  and  Percentages  of  Respondents  by  Rank  for  each  Squadron 


Figure  6.  Percentage  of  MCAS  Respondents’  with  Rank  E1-E5  for  each  Squadron 
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4.  MCAS  MOSE  Component  Statistics 

Table  8  displays  the  six  MOSE  categories  and  respective  MCAS  questions  that 
comprise  each  component.  Table  9  displays  the  mean  response  by  component  and 
squadrons  of  the  3rd  MAW.  The  3rd  MAW  question  averages  by  squadron  and  the  MOSE 
component  statistics  for  the  MAGS  and  type  aircraft  can  be  found  in  Appendices  C  and  D 
respectively. 


MOSE  Category 

Questions 

Process  Auditing  (PA) 

1,2, 3,4,  5,6 

Reward  System  &  Safety  Climate  (RS) 

7, 8, 9, 10, 11,  12, 13, 14 

Quality  Assurance  (QA) 

15, 16, 17, 18, 19, 20 

Risk  Management  (RM) 

21, 22, 23, 24,  25, 26, 27, 28, 29 

Command  &  Control  (CC) 

30,31,32,  33,  34,  35,36, 37 

Communication  /  Functional  Relationships 

38, 39, 40,41,42, 43 

Table  8.  MOSE  Categories  and  Questions 


Table  9.  Mean  MCAS  Response  by  Component  and  Squadron  of  3rd  MAW 
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Variances 


B.  MCAS  PRINCIPAL  COMPONENT  ANALYSIS 

Principal  Component  Analysis  (Hamilton,  1992)  is  performed  on  the  survey  data 
to  test  whether  the  factors  that  make  up  the  survey  account  equally  for  most  of  the 
variability  of  the  data.  Using  S-Plus  4.5,  the  command  “princomp  (MAW,  cor  =  F)”  is 
used  to  produce  the  results  of  the  principal  component  analysis.  The  Principal 
Component  is  defined  as  the  681  by  43  matrix  of  survey  responses  (one  row  per  survey 
and  one  column  per  question).  In  addition,  the  analysis  is  based  on  the  covariance  matrix 
rather  than  the  correlation  matrix.  The  covariance  matrix  is  used  since  all  questions  are 
on  a  common  scale.  Principal  Component  Analysis  is  often  used  to  reduce  the  number  of 
explanatory  variables  (questions),  however  it  is  being  used  here  to  revalidate  the  survey 
by  showing  no  reduction  is  required.  A  scree-plot  is  used  to  display  the  output  of  the 
principal  component  analysis  (see  Figure  7). 


Figure  7.  Principal  Component  Scree-Plot. 
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In  this  figure,  the  variance  contributed  by  each  component  is  plotted  and  the 
cumulative  variance  given.  Only  the  first  10  of  43  components  of  the  Principal 
Component  Analysis  are  displayed.  The  principal  component  loadings  are  the 
coefficients  of  the  principal  components  transformations.  Observe  that  the  first 
component,  which  is  the  maximum  variance  of  the  data,  explains  32.7  percent  of  the  total 
variance  of  the  survey.  As  shown  in  Appendix  E  all  questions  load  equally  across  the 
first  component.  Additionally,  with  a  large  difference  between  the  1st  and  2nd  component 
and  a  gradual  decrease  thereafter,  this  suggests  that  the  survey  is  dominated  by  the  first 
component  and  that  linear  combinations  of  the  questions  combine  for  the  remaining 
variance.  This  analysis  has  one  important  implication,  the  questions  in  the  survey  are 
contributing  equally  to  the  overall  assessment  of  maintenance  climate,  and  the  individual 
questions  and  their  interactions  are  required  to  capture  this  climate. 

C.  MCAS  MOSE  COMPONENT  ANALYSIS 

Only  squadrons  with  at  least  25  surveys  are  used  to  do  further  explanatory 
analysis,  Analysis  Of  Variance  (ANOVA)  and  Multiple  Comparison  Analysis 
(MANOVA)  on  the  MOSE  components.  Since  most  squadrons  have  a  maintainer  census 
between  90  and  150,  with  an  average  of  120,  it  is  deemed  necessary  to  capture  at  least  20 
percent  of  the  maintainers’  perceptions  in  order  to  draw  any  meaningful  conclusions  from 
the  survey.  The  data  set  is  reduced  by  85  surveys  to  596  by  eliminating  squadrons  D,  E, 
H,  J,  K,  N  and  Q.  This  reduction  eliminates  the  ability  to  compare  differences  between 
MAGs  and  type  aircraft.  Table  10  shows  the  remaining  squadrons  and  their  MOSE 
component  statistics. 
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Sqdrn 

Process 

Auditing 

Reward 

System 

Quality 

Assurance 

Risk 

Management 

Command 
&  Control 

Comms/ 
Func  Rel 

Sqdrn 

Mean 

A 

3.75 

3.41 

3.63 

3.12 

3.50 

3.43 

3.47 

B 

4.07 

3.53 

3.94 

3.47 

3.85 

3.66 

3.75 

C 

4.00 

3.34 

3.56 

3.30 

3.40 

3.37 

3.50 

F 

4.02 

3.67 

3.60 

3.65 

3.68 

3.55 

3.70 

G 

3.79 

3.37 

3.42 

2.98 

3.33 

3.21 

3.35 

I 

3.98 

3.39 

3.52 

3.28 

3.51 

3.53 

3.54 

L 

4.02 

3.74 

3.99 

3.56 

3.83 

3.92 

3.84 

M 

3.61 

3.25 

3.37 

2.99 

3.17 

3.19 

3.26 

O 

3.58 

3.21 

3.32 

2.82 

3.18 

3.20 

3.22 

P 

3.74 

3.28 

3.66 

3.20 

3.45 

3.63 

3.49 

Component 

Mean 

3.86 

3.42 

3.60 

3.24 

3.49 

3.47 

3.51 

Table  10.  Mean  Survey  Response  by  Component  and  Squadron 
for  Squadrons  with  at  least  25  Surveys 

Figures  8  and  9  are  bar  charts  showing  the  MOSE  mean  responses  of  the 
components  and  MCAS  mean  responses  for  the  squadrons,  respectively.  The  average 
Likert  scale  response  (1  =  “Strongly  Disagree,  2  =  “Disagree”,  3  =  “Neutral”,  4.  “Agree,” 
5  =  “Strongly  Agree”)  for  both  the  MOSE  components  and  the  squadrons  was  3.51.  Of 
note  is  the  mean  for  Risk  Management  (RM)  is  3.24  with  a  low  of  2.82  (O)  and  a  high  of 
3.65  (F).  The  highest  scored  component  is  Process  Auditing  (PA)  with  a  mean  of  3.86 
and  a  high  of  4.07  (B)  and  low  of  3.58  (O).  For  the  squadrons  notice  that  G,  M  and  O  are 
well  below  the  mean  while  the  high  for  all  is  3.84  (L)  and  the  low  is  3.22  (O). 


Figure  8.  MOSE  Component  Means  with  Average 
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Figure  9.  Squadron  MCAS  Means  with  Average 


1.  Analysis  of  Variance 

A  two-factor  ANOVA  is  conducted  to  determine  if  either  the  squadron  or  the 
MOSE  component  show  any  effect  on  the  mean  survey  response.  Let  7  represent  the 
levels  of  the  first  factor  (squadrons)  and  7  represent  the  levels  of  the  second  factor 
(MOSE  components)  then  in  this  data  set  7=10  (the  ten  squadrons)  and  7=6  (the  six 
MOSE  components),  and  there  are  7*7=60  possible  cross  classifications  that  make  up  the 
levels  of  the  two  factors.  The  dependent  variable  is  the  mean  question  response  given  in 
Table  10. 

Appendix  F  has  a  detailed  listing  of  the  S-PLUS  code  and  associated  output  in  the 
ANOVA  analysis  on  the  data  in  Table  10.  The  results  of  the  analysis  on  the  two  factors 
with  average  as  the  response  is  displayed  on  the  next  page  in  Table  11. 
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df 

Sum  of  Sq 

Mean  Sq 

F 

Pf-value 

Squad 

9 

2.27 

0.252 

26.7 

3.22  x  10‘15 

Comp 

5 

2.13 

45.1 

2.20  x  10’15 

Residuals 

45 

0.43 

1 

Table  11.  Analysis  of  Variance  on  Squadron  and  MOSE  Factors 


The  p-values  for  the  null  hypothesis  that  there  is  no  squadron  effect  and  there  is 
no  component  effect  are  less  than  10'14.  Thus  there  is  strong  evidence  that  at  least  one 
squadron  has  a  statistically  different  mean  score,  and  there  is  strong  evidence  that  at  least 
one  of  the  MOSE  components  do  not  yield  the  same  mean  responses. 

2.  Multiple  ANOVA  Comparisons 

A  multiple  comparison  test  using  Tukey’s  Procedure  (or  “T”  method)  is 
conducted  to  determine  which  squadrons  are  different  from  one  another.  Utilizing  the 
Studentized  Range  probability  distribution,  simultaneous  confidence  intervals  for  all 
pairwise  comparisons  were  computed  at  a  selected  level  of  significance.  In  this  case,  the 
pairwise  comparing  of  the  means  of  all  six  MOSE  components  results  in  fifteen 
comparisons.  The  resulting  confidence  intervals  are  intervals  for  the  values  of  all  pairs  of 
differences  between  true  treatment  means.  Each  interval  that  doesn’t  include  the  value  of 
zero  yields  the  conclusion  that  the  treatment  means  differ  significantly  (Devore,  1995). 

Appendix  F  has  a  detailed  listing  of  the  S-PLUS  code  and  associated  output  for 
Tukey’s  procedure.  Figure  10  is  the  boxplot  of  mean  responses  for  the  10  squadrons. 
Having  the  squadron  means  of  3.22  (0),  3.26  (M),  3.35  (G),  3.47  (A),  3.49  (P),  3.49  (C), 
3.54,  (I),  3.70  (F),  3.75  (B),  and  3.84  (L)  arranged  in  increasing  order,  the  graphical 
display  of  the  confidence  intervals  (Figure  1 1)  is  used  to  obtain  those  pairings  that 
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contain  a  value  of  zero.  The  results  of  this  comparison  is  summarized  graphically  in 
Figure  12.  Any  squadrons  not  underscored  by  the  same  line,  are  statistically  different 
from  each  other  at  a  five  percent  level  of  significance.  Figure  12  shows  that  O  and  M  are 
different  from  all  other  squadrons  except  G.  Squadrons  A,  P  and  C  are  different  from  O, 
M  and  F,  B,  and  L.  Finally,  squadrons  B  and  L  are  different  from  all  other  squadrons 
except  F.  Since  G  and  F  form  the  bridge  between  the  three  groups  they  are  considered 
boundary  squadrons  having  characteristics  of  two  groups. 


Figure  10.  Boxplots  of  Squadrons 
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Figure  11.  Tukey’s  Pairwise  Comparison  of  Squadrons 
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Figure  12.  Identifying  Statistically  Different  Squadrons 


Finally,  a  multiple  comparison  analysis  is  conducted  on  the  MOSE  component 
factor  in  the  same  manner  as  above.  Again,  having  the  MOSE  component  means  of  3.24 
(RM),  3.42  (RS),  3.47  (CF),  3.49  (CC),  3.60  (QA),  and  3.86  (PA)  arranged  in  increasing 
order,  the  graphical  display  of  the  confidence  intervals  (Figure  13)  is  used  to  obtain  those 
pairing  that  contain  a  value  of  zero.  Those  squadrons  are  then  underlined  as  shown  in 
Figure  14.  Figure  14  shows  that  RM  and  PA  are  found  to  be  different  from  all  other 
components.  QA  is  found  to  be  different  from  all  components  except  CC. 


response  variable:  Squad 

Figure  13.  Tukey’s  Pairwise  Comparison  of  MOSE  Components 
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RM  RS  CF  CC  QA  PA 


Figure  14.  Identifying  Statistically  Different  MOSE  Components 


D.  HFACS-ME  DESCRIPTIVE  STATISTICS 

Between  calendar  years  1998  -  1999  the  3rd  MAW  experienced  two  class  A, 
three  class  B,  four  class  C  Maintenance  Related  Incidents  (MRIs)  13  HAZREPS.  Table 
12  lists  the  MRMs  with  the  associated  cost  of  the  mishap  or  HAZREP.  During  this  time 
frame,  squadrons  D,  L,  M,  P  and  Q  reported  no  MRMs  or  HAZREPS. 


Class  A 

Class  B 

Class  C 

HAZREP 

D 

$10,000 

B 

C 

$11,783,499 

E 

$227,350 

$360,515 

$195,730 

$10,000 

$10,000 

a 

$24,397,330 

$10,000 

$10,000 

o 

$10,000 

H 

$10,000 

I 

$10,000 

$10,000 

J 

$10,000 

K 

$116,751 

$77,005 

$10,000 

$10,000 

N 

$10,000 

0 

$120,487 

Table  12.  Class  A,  B  and  C  MRMs  and  HAZREPS  for  the  3rd  MAW 
between  Jan  1998  -  Dec  1999 


Using  HFACS-ME  to  identify  the  casual  factors  in  the  MRIs  and  HAZREPS  the 
total  first  and  second  order  casual  factors  for  each  squadron  reporting  an  incident  are 
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counted  and  are  presented  in  Table  13.  If  there  is  no  second  order  casual  factor  observed 
that  column  is  removed. 


MA 

MC 

sc 

WC 

Error 

Violation 

Crew  Coord 

Unforeseen 

Sqdrn 

Equipment 

A 

i 

i 

B 

i 

i 

i 

2 

C 

2 

1 

D 

E 

5 

i 

4 

1 

i 

F 

2 

2 

1 

G 

1 

I 

1 

1 

1 

2 

J 

1 

i 

K 

4 

1 

3 

N 

1 

O 

1 

1 

3 

1 

Table  13.  HFACS-ME  1st  and  2nd  Order  Casual  Factors  for  MRIs  and 
HAZREPS  of  the  3rd  MAW  between  Jan  1998  -  Dec  1999 


E.  HFACS-ME  /  MCAS  CORRELATION  ANALYSIS 

Finally,  an  effort  is  undertaken  to  identify  relationships  between  the  average 
MCAS  score  for  a  squadron  and  their  MRIs  and  HAZREPS.  This  is  accomplished  by 
first  combining  all  MRIs  and  HAZREPS  for  each  squadron,  and  then  dividing  by  the  total 
flight  hours  for  the  past  two  years,  giving  an  incident  rate  per  flight  hour.  The  summary 
of  this  computation  along  with  mean  MCAS  response  for  each  squadron  is  given  in  Table 
14.  A  simple  linear  regression  of  MCAS  mean  response  versus  incident  rate  from  Table 
14  is  performed.  Note  that,  only  those  squadrons  who  completed  25  or  more  surveys  are 
included.  Figure  15  plots  the  linear  regression  with  mean  MCAS  as  the  predictor 
variable  and  incident  rate  per  flight  hour  as  the  response  variable.  Squadron  F  can  be 
observed  as  an  influential  point  greatly  affecting  the  slope  coefficient  in  the  regression. 
Figure  16  gives  the  same  linear  regression  with  squadron  F  removed.  Appendix  G 
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contains  the  detailed  results  of  both  regressions.  For  the  model  containing  F  the 
estimated  slope  coefficient  is  32.8  with  an  associated  t-value  of  .672.  For  the  model 
without  F  the  estimated  slope  coefficient  is  -8.71  with  an  associated  t  value  of  -.395; 
neither  gives  slopes  that  are  statistically  different  from  zero.  Thus,  the  results  are 
inconclusive.  There  is  not  enough  evidence  to  say  that  there  is  a  relationship  (either 
positive  or  negative)  between  mean  MCAS  response  and  incident  rate.  Similar  results  are 
obtained  when  incidents  are  weighted  according  to  cost. 


#  Incidents 

Flight  Hours  (FH) 

Incident  /  FH 

Mean  MCAS 

A 

i 

7257.2 

13.8 

3.47 

B 

1 

12076.6 

8.3 

3.75 

C 

i 

6403.9 

15.6 

3.49 

F 

3 

3128.9 

95.9 

3.69 

G 

1 

6209.3 

16.1 

3.35 

I 

2 

5246.5 

38.1 

3.53 

L 

0 

2378.2 

0.0 

3.84 

M 

0 

3052.9 

0.0 

3.26 

O 

1 

6402 

15.6 

3.22 

P 

0 

5783.2 

0.0 

3.49 

Table  14. 3rd  MAW  Incident,  Flight  Hour,  Incident  Rate  and 
Mean  MCAS  data  from  Jan  1998  -  Dec  1999 
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Incident  Rate/FH  Incident  Rate/FH 


Figure  15.  Simple  Linear  Regression  with  Mean  MCAS  as 
Predictor  and  Incident  Rate/FH  as  Response 


3.10  3.20  3.30  3.40  3.50  3.60  3.70  3.80  3.90 

Mean  MCAS 

Figure  16.  Simple  Linear  Regression  with  Mean  MCAS  as 
Predictor  and  Incident  Rate/FH  as  Response  minus  F 
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Table  15  summarizes  an  ad  hoc  analysis  done  by  the  School  of  Aviation  Safety  at 
the  Naval  Postgraduate  School  on  the  10  MRIs  of  the  3rd  MAW  and  the  MCAS  MOSE 
components.  In  this  analysis,  two  people  with  expertise  in  general  aviation  and  aviation 
safety  review  10  MRIs.  In  Table  15,  an  “X”  in  the  cell  indicates  that  at  least  one  question 
in  a  particular  component  addresses  the  casual  factors  associated  with  the  incident  and 
that  both  persons  evaluating  the  MRI  at  the  School  of  Aviation  Safety  at  the  Naval 
Postgraduate  School  agree  that  the  incident  has  that  MOSE  component  associated  with  at 
least  one  of  the  causal  factors  in  the  incident.  For  example,  the  MOSE  component,  PA, 
has  at  least  one  question  that  addresses  a  casual  factor  in  incidents  “1”,  “2”,  “4”,  and  “6” 
and  incident  “1”  has  a  PA  and  CF  MOSE  component  in  the  causal  factors. 

This  table  indicates  that  based  on  expert  judgement,  there  appears  to  be  a 
relationship  between  the  MOSE  components  of  the  MCAS  and  the  MRIs.  Every  incident 
has  at  least  one  causal  factor  associated  with  a  MOSE  component.  Of  note  is  the  high 
occurrence  of  the  MOSE  component  QA  showing  up  in  eight  out  of  ten  incidents. 
Likewise,  the  MOSE  component  RS  has  a  low  occurrence  showing  up  in  only  one  out  of 
ten  incidents. 


INCIDENTS  i 

MOSE 

COMPONENTS 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

PA 

X 

X 

X 

X 

RS 

X 

QA 

X 

X 

X 

X 

X 

X 

X 

X 

RM 

X 

X 

CC 

X 

X 

X 

X 

CF 

X 

X 

Table  15. 3rd  MAW  Incident  Factors  Captured  by  MCAS  MOSE  Components 
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V.  SUMMARY,  CONCLUSION,  AND  RECOMMENDATIONS 


A.  SUMMARY 

Squadron  commanders  have  a  fundamental  responsibility  to  safeguard  personnel 
and  material  resources  while  at  the  same  time  minimizing  risk  while  accomplishing 
assigned  missions.  Dramatic  strides  in  reducing  the  Class  A  Flight  Mishap  (FM)  rate 
have  occurred  and  only  recently  have  these  reductions  begun  to  level  off  forcing  Naval 
Aviation  to  look  at  alternate  ways  to  continue  the  reduction.  Recognizing  that  one  in  five 
Naval  Aviation  Class  A  FMs  has  maintenance-related  casual  factors  a  recent  effort  has 
begun  to  address  aircraft  maintenance  issues  in  order  to  continue  the  Class  A  FM 
reductions.  Additionally  major  and  minor  mishaps  have  come  under  review  and  for  this 
study  were  called  maintenance-related  incidences  (MRIs). 

The  primary  focus  of  this  study  was  to  analyze  the  Maintenance  Climate 
Assessment  Survey  (MCAS)  and  its  ability  to  capture  maintainer  perceptions  on  3rd 
Marine  Air  Wing  (MAW)  maintenance  operations.  Of  secondary  interest,  was  the 
application  of  the  Human  Factors  Accident  Classification  System  -  Maintenance 
Extension  (HFACS-ME)  to  the  3rd  MAW  to  explore  their  human  errors  and  to  assess  if 
there  is  a  relationship  between  their  MRIs  and  the  MCAS.  Both  of  these  vehicles  were 
looked  at  in  an  effort  to:  1)  obtain  3rd  MAW  maintainer  perception  of  maintenance 
operations  safety,  2)  evaluate  the  ability  of  the  MCAS  to  detect  differences  in  squadron 
maintenance  safety  climate  and  how  it  is  associated  with  the  MRI  histories  and  3) 
determine  if  human  error  is  a  significant  factor  in  3rd  MAW  MRIs  using  HFACS-ME. 
This  research  involved  the  collection  of  977  survey  results  during  the  first  part  of  Fiscal 
Year  (FY)  2000  and  the  analysis  of  21  MRIs  from  January  1998  to  December  1999. 
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B.  CONCLUSIONS 


The  results  of  this  study  show  that  the  MCAS  can  be  utilized  by  Naval  Aviation 
as  a  tool  for  effectively  capturing  a  maintainer’s  perceptions  of  safety  in  maintenance 
operations.  Principal  Component  Analysis  fails  to  identify  any  single  MOSE  component 
or  question  that  was  responsible  for  controlling  the  outcome  of  the  survey.  Through 
analysis  of  the  removed  surveys  with  missing  values  or  the  same  response  to  all  43 
questions  it  was  determined  that  question  order  effects  were  present.  One  minor  finding 
was  Question  21  was  found  to  not  load  well  in  the  first  principal  component  due  to  the 
negative  wording  in  the  question. 

Additionally,  through  Analysis  of  Variance  (ANOVA)  and  multiple  comparison 
testing,  it  is  determined  that  the  MCAS  is  able  to  detect  differences  between  both  the 
squadron’s  mean  MCAS  and  MOSE  component  response.  Through  this  analysis,  it  was 
discovered  that  Risk  Management  is  a  concern  within  the  3rd  MAW.  It  also  highlighted  a 
positive  outlook  with  respect  to  the  command  Process  Auditing.  Unfortunately,  this 
study  was  unable  to  conduct  the  same  analysis  at  the  Marine  Air  Group  or  Type  Aircraft 
level  due  to  an  insufficient  number  of  surveys  across  these  levels  of  the  3rd  MAW. 

Finally,  the  analysis  done  on  the  Maintenance  Related  Incidences  (MRIs)  of  the 
3rd  MAW  using  HFACS-ME  was  also  limited  by  the  number  of  surveys  as  well  as  a 
general  low  rate  of  MRIs.  The  fact  that  the  safety  climate  of  a  squadron  lasts  about  two 
years  makes  it  impossible  to  conduct  an  in-depth  analysis  of  the  ordered  causal  factors 
relating  to  the  MRIs  in  a  one  time  study.  The  low  t-value  associated  with  the  linear 
regression  fit  with  squadron  F  included  and  removed  lead  one  to  conclude  that  this  one 
study  is  insufficient  to  demonstrate  a  relationship  between  the  two.  However,  based  on 
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the  content  analysis  of  the  MRIs  and  the  MCAS  MOSE  components  it  is  clear  that  the 
survey  is  addressing  issues  related  to  incidents  of  the  3rd  MAW.  Of  note  is  the  low  rate 
of  the  Reward  System  and  Safety  Climate  (RS)  component  in  the  incidents  of  the  3rd 
MAW  but  this  could  simply  be  due  to  chance  with  such  a  relatively  low  number  of 
incidents. 

C.  RECOMMENDATIONS 

Based  on  conclusions  from  the  study,  a  number  of  recommendations  concerning 
question  order,  minor  MCAS  modifications  and  administration  and  analysis  of  the  survey 
are  listed  below: 

1.  The  3rd  MAW  should  review  their  Risk  Management  maintenance  processes. 
In  addition  this  study  can  be  used  as  a  starting  point  for  year  over  year 
comparison  of  3rd  MAW  MCAS  and  MRI  data  to  determine  to  what  degree 
there  is  a  relationship  between  perceptions  of  maintenance  safety  and  actual 
safety  records 

2.  Minor  modifications  of  the  MCAS  need  to  be  made  including,  1)  Positively 
wording  question  21  2)  randomizing  the  questions  with  respect  to  the  MOSE 
components  to  reduce  response  order  effects  and  3)  revisiting  the  questions  of 
the  Reward  System  component  to  ensure  they  are  adequately  addressing 
casual  factors  in  MRIs. 

3.  Since  often  times  the  survey  was  given  under  different  circumstances,  a  more 
systematic  administration  of  the  MCAS  needs  to  be  developed.  One  method 
currently  being  developed  by  the  School  of  Aviation  Safety  at  the  Naval 
Postgraduate  School  is  on-line  version  of  the  MCAS  similar  to  the  Command 
Safety  Assessment  (CSA)  currently  being  given  to  aircrew.  This  method  has 
the  advantage  of  real  time  analysis  of  surveys  from  a  command. 

Implementation  of  these  recommendations  would  lead  to  a  more  effective  and  useful 

MCAS  for  use  in  the  future  by  other  aviation  commands.  In  addition,  as  previous 

research  has  shown  intervention  in  the  Risk  Management  practices  of  the  3rd  MAW  could 

lead  to  substantial  reductions  in  the  MRIs  associated  with  this  casual  factor. 


47 


While  the  above  recommendations  will  positively  affect  the  MCAS,  further 
research  is  needed  in  the  following  areas: 

1.  In  order  to  reduce  the  response  error  that  may  be  introduced  by  a  persons 
background  it  is  recommend  that  a  study  be  undertaken  to  asses  the 
relationship,  if  any,  between  a  persons  demographic  and  MCAS  responses. 
To  completely  validate  this  survey,  it  needs  to  be  shown  that  the  MCAS 
responses  are  independent  of  the  demographics  of  the  respondent. 

2.  Finally,  to  accurately  assess  the  link  between  the  MCAS  and  MRI  rate  a 
systematic  study  of  squadrons  and  their  mishap  rate  needs  to  be  conducted. 

The  benefits  to  this  additional  research,  will  be  to  make  the  MCAS  a  more  effective  tool 

in  capturing  maintainer  perceptions  on  maintenance  operations  while  continuing  to  strive 

for  the  ultimate  safety  goal  of  intervention  by  aviation  commanders  in  real  time  to  correct 

safety  deficiencies  that  will  eventually  lead  to  a  future  incident. 
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APPENDIX  A.  43-ITEM  MAINTENACE  CLIMATE  ASSESSMENT  SURVEY 

MAINTENANCE  CLIMATE  ASSESSMENT  SURVEY  fMCASt 

Purpose;  The  MCAS  was  designed  to  capture  maintainer  perceptions  of  maintenance  operations  as  they 
relate  to  safety.  Your  responses  help  guide  Naval  Aviation’s  on-going  efforts  to  reduce  aviation  related 
mishaps.  Thank  vou  in  advance  for  vour  participation! 

Directions:  Do  not  write  on  this  form.  Fill  in  all  of  your  responses  using  the  computer  sheet  provided. 
Fill  in  each  box  that  corresponds  to  your  response  completely  using  a  pencil.  This  is  not  a  timed  event,  so 
answer  each  question  carefully  and  honestly.  Individual  responses  will  not  be  reported,  only  compiled 
results  will  be  provided  to  each  squadron. 

Parti-  Demographics  has  six  items  requesting  unit  and  biographical  data.  This  information  will  aid  in  the 
response  analysis.  NO  attempts  will  be  made  to  identify  individuals. 

Partll-  Perceptions  has  43  questions  pertaining  to  the  maintenance  operations.  Please  chose  the  response 
to  each  item  that  most  correctly  reflects  your  honest  opinion.  Responses  are: 

A- Strongly  Agree  B- Agree  C- Neutral  D- Disagree  E- Strongly  Disagree 
Part  I-  Demographics 

Line  1  Fill  in  the  numbered  circle  corresponding  to  your  community? 

VMGR  (1)  VMA  (2)  VMFA  (3)  HMT  (4)  HMM(5) 
VMAQ  (6)  HMH  (7)  VMH  (9)  Other  (9)  ( ) 

Line  2-4  Fill  in  the  circles  corresponding  to  your  squadron  number 

Line  5  Fill  in  the  numbered  circle  corresponding  with  you  rank 

(1)  El-3  (2)E4-5  (3)E6-7  (4)E8-9  (5)W01-4  (6)01-03  (7)04-5 

Line  6  Fill  in  the  numbered  circle  corresponding  to  your  total  years 

of  Aviation  Maintenance  experience 
(1)  <1  (2)  1-2  (3)3-5  (4)6-10  (5)11-15  (6)15-20  (7)20+ 

Line  7  Fill  in  the  numbered  circle  corresponding  to  your  work  center 

(l)Powerplants  (2)Airframes  (3)Survival  (4)  Quality  Assurance 
(5)Ordnance  (6)Avionics  (7)MAINT  Control  (8)Line  (9)Other 

Line  8  Fill  in  the  numbered  circle  corresponding  to  your  shift 

(l)Day  (2)Night 

Part  II  Perceptions 

Fill  in  the  lettered  circle  that  corresponds  with  your  response  to  each  item. 


SA 

A 

N 

D 

SD 

1 .  The  command  adequately  reviews  and  updates  safety. 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

2.  The  command  monitors  maintainer  qualifications  and  has 

(A) 

(B) 

(C) 

(D) 

(E) 

a  program  that  targets  training  deficiencies. 

SA 

A 

N 

D 

SD 

3.  The  command  uses  safety  and  medical  staff  to  identify/ 

(A) 

(B) 

(C) 

(D) 

(E) 

manage  personnel  at  risk. 

SA 

A 

N 

D 

SD 

4.  CDIs/QARs  routinely  monitor  maintenance  evolutions. 

(A) 

(B) 

(C) 

(D) 

(E) 
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SA 

A 

N 

D 

SD 

5.  Tool  control  is  taken  seriously  in  the  command  and 

(A) 

(B) 

(C) 

(D) 

(E) 

support  equipment  licensing  is  closely  monitored. 

SA 

A 

N 

D 

SD 

6.  Signing  off  personnel  qualifications  are  taken  seriously. 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

7.  Our  command  climate  promotes  safe  maintenance. 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

8.  Supervisors  discourage  SOP,  NAMP,  or  other  procedure 

(A) 

(B) 

(C) 

(D) 

(E) 

violations  and  encourage  reporting  safety  concerns. 

SA 

A 

N 

D 

SD 

9.  Peer  influence  discourages  SOP,  NAMP,  or  other 

(A) 

(B) 

fC) 

(D) 

violations  and  individuals  feel  free  to  report  them. 

SA 

A 

N 

D 

SD 

10.  Violations  of  SOP,  NAMP,  or  other  procedures  are  not 

(A) 

(B) 

(C) 

(D) 

(E) 

common  in  this  command. 

SA 

A 

N 

D 

SD 

1 1.  The  command  recognizes  individual  safety  achievement 

(A) 

(B) 

(C) 

(D) 

(E) 

through  rewards  and  incentives. 

SA 

A 

N 

D 

SD 

12.  Personnel  are  comfortable  approaching  supervisors 

(A) 

(B) 

(C) 

(D) 

(E) 

about  personal  problems/illness 

SA 

A 

N 

D 

SD 

13.  Safety  NCO,  QAR,  and  CDI,  are  sought  after  billets. 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

14.  Unprofessional  behavior  is  not  tolerated  in  the  command 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

15.  The  command  has  a  reputation  for  quality  maintenance 

(A) 

(B) 

(C) 

(D) 

(E) 

and  sets  standards  to  maintain  quality  control. 

SA 

A 

N 

D 

SD 

16.  QA  and  Safety  are  well  respected,  and  are  seen  as 

(A) 

(B) 

(C) 

(D) 

(E) 

essential  to  mission  accomplishment. 

SA 

A 

N 

D 

SD 

1 7.  QARs/CDIs  sign-off  after  required  actions  are  complete 

(A) 

(B) 

(C) 

(D) 

(E) 

and  are  not  pressured  by  supervisors  to  sign-off. 

SA 

A 

N 

D 

SD 

1 8.  Maintenance  on  detachments  is  of  the  same  quality 

(A) 

(B) 

(C) 

(D) 

(E) 

as  that  at  home  station. 

SA 

A 

N 

D 

SD 

19.  Required  publications/tools/equipment  are  available, 

(A) 

(B) 

(C) 

(D) 

(E) 

current/serviceable,  and  used. 

SA 

A 

N 

D 

SD 

20.  QARs  are  helpful,  and  QA  is  not  “feared”  in  my  unit. 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

21.  Multiple  job  assignments  and  collateral  duties  adversely 

(A) 

(B) 

(C) 

(D) 

(E) 

affect  maintenance. 

SA 

A 

N 

D 

SD 

22.  Safety  is  part  of  maintenance  planning,  and  additional 

(A) 

(B) 

(C) 

(D) 

(E) 

training/support  is  provided  as  needed. 
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23.  Supervisors  recognize  unsafe  conditions  and  manage 
hazards  associated  with  maintenance  and  the  flight-line. 

24.  I  am  provided  adequate  resources,  time,  personnel  to 
accomplish  my  job. 

25.  Personnel  turnover  does  not  negatively  impact  the 
command’s  ability  to  operate  safely. 

26.  Supervisors  are  more  concerned  with  safe  maintenance 
than  the  flight  schedule,  and  do  no  permit  cutting  comers. 

27.  Day/Night  Check  have  equal  workloads,  and  staffing  is 
sufficient  on  each  shift. 

28.  Supervisors  shield  personnel  from  outside  pressures  and 
are  aware  of  individual  workload. 

29.  Based  upon  my  command’s  current  assets/manning  it 
is  not  over-committed. 

30.  My  command  temporarily  restricts  maintainers  who  are 
having  a  problems. 

3 1 .  Safety  decisions  are  made  at  the  proper  levels  and 
work  center  supervisor  decisions  are  respected. 

32.  Supervisors  communicate  command  safety  goals  and 
are  actively  engaged  in  the  safety  program. 

33.  Supervisors  set  the  example  for  following  to 
maintenance  standards  and  ensure  compliance. 

34.  In  my  command  safety  is  a  key  part  of  all  maintenance 
operations,  and  all  are  responsible/accountable  for  safety. 

35.  Safety  education  and  training  are  comprehensive  and 
effective. 

36.  All  maintenance  evolutions  are  properly  briefed, 
supervised,  and  staffed  by  qualified  personnel. 

37.  Maintenance  Control  is  effective  in  managing  all 
maintenance  activities. 

38.  Good  communication  exists  up/down  the  chain  of 
command. 

39.  I  get  all  the  information  I  need  to  do  my  job  safely. 


40.  Work  center  supervisors  coordinate  their  actions. 


4 1 .  My  command  has  effective  pass-down  between  shifts. 
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SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N  . 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

42.  Maintenance  Control  troubleshoots/resolves  gripes  before 
flight. 

43.  Maintained  are  briefed  on  potential  hazards  associated 
with  maintenance  activities. 


SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 
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APPENDIX  B.  MODEL  OF  SAFETY  EFFECTIVENESS  COMPONENTS 


COMPONENT  1:  PROCESS  AUDITING 

1 .  The  command  adequately  reviews  and  updates  safety  practices. 

2.  The  command  monitors  maintainer  qualifications  and  has  a  program  that  targets 
training  deficiencies. 

3.  The  command  uses  safety  and  medical  staff  to  identify/manage  personnel  at  risk. 

4.  CDIs/QARs  routinely  monitor  maintenance  evolutions. 

5.  Tool  Control  is  taken  seriously  in  the  command  and  support  equipment  licensing  is 
closely  monitored. 

6.  Signing  personal  qualifications  are  taken  seriously. 

COMPONENT  2:  REWARD  SYSTEM  and  SAFETY  CLIMATE 

1 .  Our  command  climate  promotes  safe  maintenance  and  flight  operations. 

2.  Supervisors  discourage  SOP,  NAMP  or  other  procedure  violations  and  encourage 
reporting  safety  concern. 

3.  Peer  influence  discourages  SOP,  NAMP  or  other  violations  and  individuals  feel  free 
to  report  them. 

4.  Violations  of  SOP,  NAMP  or  other  procedures  are  not  common  in  this  command. 

5.  The  command  recognizes  individual  safety  achievement  through  rewards  and 
incentives. 

6.  Personnel  are  comfortable  approaching  supervisors  about  personal  problems/illness. 

7.  Safety  NCO,  QAR,  and  CDI,  are  sought  after  billets. 

8.  Unprofessional  behavior  is  not  tolerated  in  the  command. 

COMPONENT  3:  QUALITY  ASSURANCE 

1 .  The  command  has  a  reputation  for  quality  maintenance  and  has  set  standards  to 
maintain  quality  control. 

2.  QA  and  Safety  are  well  respected,  and  are  seen  as  essential  to  mission 
accomplishment. 

3 .  Q  ARs/CDIs  sign-off  after  required  actions  are  complete  and  are  not  pressured  by 
supervisors  to  sign-off. 

4.  Maintenance  on  detachments  is  the  same  quality  as  that  at  home  station. 

5.  Required  publications/tools/equipment  are  available,  current/serviceable,  and  used. 

6.  QARs  are  helpful,  and  QA  is  not  “feared”  in  my  unit. 
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COMPONENT  4:  RISK  MANAGEMENT 


1 .  Multiple  job  assignments  and  collateral  duties  adversely  affect  maintenance. 

2.  Safety  is  part  of  maintenance  planning,  and  additional  training/support  is  provided  as 
needed. 

3 .  Supervisors  recognize  unsafe  conditions  and  manage  hazards  associated  with 
maintenance  and  the  flight  line. 

4.  I  am  provided  adequate  resources,  time,  personnel  to  accomplish  my  job. 

5.  Personnel  turnover  does  not  negatively  impact  the  command’s  ability  to  operate 
safely. 

6.  Supervisors  are  more  concerned  with  safe  maintenance  than  the  flight  schedule,  and 
do  not  permit  cutting  comers. 

7 .  Day/Night  Check  have  equal  workloads,  and  staffing  is  sufficient  on  each  shift. 

8.  Supervisors  shield  personnel  from  outside  pressures  and  are  aware  of  individual 
workload. 

9.  Based  upon  my  command’s  current  assets/manning  it  is  not  over-committed. 

COMPONENT  5:  COMMAND  AND  CONTROL 

1 .  My  command  temporarily  restricts  maintained  who  are  having  problems. 

2.  Safety  decisions  are  made  at  the  proper  levels,  work  center  supervisors  decisions  are 
respected. 

3 .  Supervisors  communicate  command  safety  goals  and  are  actively  engaged  in  the 
safety  program. 

4.  Supervisors  set  the  example  for  following  to  maintenance  standards  and  ensure 
compliance. 

5.  In  my  command  safety  is  a  key  part  of  all  maintenance  operations  and  all  are 
responsible/accountable  for  safety. 

6.  Safety  education  and  training  are  comprehensive  and  effective. 

7 .  All  maintenance  evolutions  are  properly  briefed,  supervised,  and  staffed  by  qualified 
personnel. 

8.  Maintenance  control  is  effective  in  managing  all  maintenance  activities. 

COMPONENT  6:  COMMUNICATION  /  FUNCTIONAL  RELATIONSHIPS 

1 .  Good  communication  exists  up/down  the  chain  of  command. 

2.  I  get  all  the  information  I  need  to  do  my  job  safely. 

3 .  Work  center  supervisors  coordinate  their  actions. 

4.  My  command  has  effective  pass-down  between  shifts. 

5 .  Maintenance  Control  troubleshoots/resolves  gripes  before  flight. 

6.  Maintainers  are  briefed  on  potential  hazards  associated  with  maintenance  activities. 


APPENDIX  C.  3RD  MAW  QUESTION  AVERAGES  BY  SQUADRON 


Sqdrn 

Ql 

Q2 

Q3 

Q4 

Q5 

Q6 

Q7 

Q8 

Q9 

foio 

Qll 

Q12 

Q13 

Q14 

Q15 

A 

3.76 

3.57 

3.43 

4.05 

3.81 

3.86 

3.86 

3.38 

3.29 

2.90 

3.81 

3.33 

3.43 

3.76 

B 

4.10 

3.38 

4.33 

3.95 

3.05 

3.81 

3.19 

3.57 

4.24 

C 

3.86 

3.19 

4.38 

4.33 

4.10 

4.00 

3.43 

3.05 

3.43 

2.48 

3.33 

3.48 

3.52 

3.76 

D 

3.90 

3.48 

3.24 

3.90 

3.62 

HErl 

3.43 

3.71 

3.38 

3.00 

3.90 

■9 

HB3 

EWtt 

3.50 

3.88 

fclrld 

3.13 

4.00 

3.38 

4.25 

4.50 

F 

EB3 

3.90 

3.76 

Pld 

4.14 

3.86 

3.90 

3.29 

4.00 

3.76 

3.38 

3.81 

G 

4.16 

3.76 

3.14 

4.10 

3.81 

3.76 

3.62 

4.05 

H 

4.42 

3.89 

3.11 

4.26 

3.89 

3.74 

4.05 

3.79 

3.37 

3.68 

3.68 

4.53 

I 

4.14 

4.24 

4.05 

4.05 

3.52 

3.57 

4.24 

J 

4.22 

3.78 

3.11 

Pill 

4.22 

4.33 

4.22 

4.11 

3.56 

4.00 

4.44 

K 

3.73 

3.73 

3.18 

S3PH 

4.09 

3.00 

3.00 

3.18 

3.00 

3.18 

3.64 

2.91 

EE3 

4.10 

3.86 

3.86 

3.90 

3.52 

3.76 

3.05 

3.81 

4.00 

4.05 

4.48 

BSfl 

3.62 

2.76 

3.76 

4.19 

3.57 

3.57 

3.43 

3.10 

3.00 

2.81 

3.38 

3.48 

3.24 

3.71 

N 

3.44 

3.67 

3.78 

3.67 

3.56 

3.67 

3.78 

O 

3.62 

3.43 

3.86 

m 

Eld 

3.43 

3.29 

3.05 

3.52 

3.62 

P 

3.84 

3.19 

2.95 

3.52 

2.38 

3.71 

sun 

3.48 

3.95 

Q 

4.13 

3.63 

3.75 

4.50 

4.13 

3.25 

4.00 

3.25 

4.38 

3.63 

AVG 

4.01 

3.74 

3.33 

4.10 

3.94 

3.79 

3.83 

3.65 

3.38 

3.45 

2.92 

3.62 

3.35 

3.65 

3.96 

kfrrirfrri 

m 

EXE3 

EE3 

EXE 

EH 0 

E2D 

EXE 

Q23 

Q24 

Q25 

Q26 

Q27 

Q28 

Q29 

Q30 

A 

EI31 

3.71 

aaa 

nag 

ns 

3.81 

2.62 

2.95 

2.05 

3.33 

3.10 

2.90 

B 

4.29 

4.19 

3.57 

3.43 

3.95 

3.43 

as 

4.19 

3.38 

ESS 

mam 

msi 

PEW 

flwa 

2.95 

3.48 

2.14 

3.19 

3.14 

2.62 

H3I 

flEH 

3.76 

3.76 

EES 

2.81 

2.95 

3.19 

E 

3.63 

4.13 

3.25 

2.88 

2.88 

3.13 

F 

mu 

3.43 

3.33 

3.81 

3.67 

3.90 

3.71 

3.10 

3.43 

3.43 

3.43 

G 

3.38 

3.24 

3.05 

3.14 

3.33 

sm 

3.00 

2.76 

2.57 

2.76 

3.14 

H 

3.74 

3.63 

3.47 

3.47 

proa 

4.05 

3.68 

ETiTTl 

3.53 

3.58 

3.26 

I 

3.67 

3.48 

fc'EU 

2.76 

3.43 

hers 

4.24 

3.29 

ms 

2.95 

2.62 

u 

3.56 

4.00 

4.00 

4.00 

3.00 

3.33 

3.78 

3.56 

3.11 

2.89 

3.27 

3.36 

3.64 

3.91 

3.09 

4.00 

3.91 

2.73 

EKH 

2.64 

3.09 

4.10 

3.90 

4.05 

4.05 

3.52 

3.48 

3.62 

3.38 

HEHI 

3.05 

3.57 

3.29 

2.86 

2.24 

2.86 

2.90 

2.86 

3.33 

3.33 

3.89 

2.89 

3.00 

3.00 

2.56 

3.78 

2.56 

3.22 

o 

3.71 

3.00 

3.52 

2.81 

3.24 

3.38 

3.10 

3.62 

2.52 

2.81 

2.38 

3.29 

p 

3.19 

3.95 

3.52 

tarn 

3.86 

3.67 

2.90 

3.14 

2.86 

2.95 

3.00 

2.81 

3.05 

Q 

4.25 

3.88 

EV«T1 

MtTu 

3.75 

3.75 

3.50 

3.25 

AVG 

3.67 

3.62 

3.56 

3.46 

3.56 

3.58 

3.69 

3.92 

3.07 

3.24 

3.29 

2.68 

3.31 

3.01 

3.08 
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Sqdrn 

Q31 

Q32 

Q33 

Q34 

Q35 

Q36 

less 

IESE 

Q40 

041 

Q42 

043 

IEEQ 

3.81 

3.57 

3.14 

3.43 

2.57 

3.71 

3.33 

3.71 

3.67 

3.57 

B 

4.29 

3.95 

4.24 

4.10 

4.05 

3.67 

3.62 

3.24 

3.86 

3.67 

3.62 

C 

3.81 

3.76 

3.43 

4.05 

3.76 

3.14 

2.67 

2.95 

3.62 

3.57 

3.81 

3.00 

3.24 

D 

3.67 

3.81 

3.33 

3.14 

3.38 

3.52 

IflEEI 

EH2 

B 

3.67 

E 

3.75 

3.63 

3.88 

Wm 

3.13 

3.63 

IHSH 

in 

HETil 

3.88 

F 

3.48 

3.67 

3.71 

ggim 

i^Em 

3.67 

3.33 

3.29 

3.67 

G 

ws 

3.71 

3.43 

2.95 

3.00 

2.43 

3.24 

Hftgl 

ETEI 

H 

4.00 

3.79 

EH 

4.00 

3.79 

3.53 

3.53 

3.58 

3.89 

EE1 

El%] 

I 

3.33 

3.71 

3.90 

3.52 

3.86 

3.29 

H23 

■EH 

4.00 

4.11 

Eafl 

4.11 

EISA 

4.22 

3.89 

3.89 

SH 

naa 

maa 

m 

2.91 

3.27 

3.73 

HH3 

2.73 

2.82 

3.45 

EltTil 

as 

3.45 

HUH 

4.05 

4.14 

EliH 

EPlH 

3.86 

BUI 

4.10 

3.95 

3.90 

3.81 

— 

EIH 

3.52 

N 

EES 

3.78 

3.33| 

3.11 

3.33 

O 

3.48 

3.48 

mm 

HH9I 

EUai 

SHI 

3.48 

3.38 

3.10 

HEH 

H!E1 

3.81 

3.71 

3.48 

3.81 

3.95 

3.43 

mu 

Q 

3.75 

3.88 

3.88 

4.13 

3.88 

3.88 

3.75 

3.50 

3.75 

4.13 

3.88 

3.50 

3.75 

AVG 

3.66 

3.68 

3.71 

3.83 

3.71 

3.39 

3.24 

3.12 

3.65 

3.61 

3.63 

3.40 

3.62 
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APPENDIX  D.  3RD  MAW  MAG  &  TYPE  AIRCRAFT  MOSE  COMPONENTS 


MAGS 

Process 

Auditing 

Reward 

System 

Quality 

Assurance 

Risk 

Manage 

Command 
&  Control 

Comms/ 
Func  Rel 

MAG 

Mean 

W 

3.79 

3.58 

3.70 

3.45 

3.63 

3.57 

3.62 

X 

3.87 

3.55 

3.65 

3.46 

3.62 

3.58 

3.62 

Y 

3.65 

3.24 

3.48 

3.52 

3.53 

Z 

3.36 

3.59 

3.15 

3.45 

3.41 

3.46 

Component 

Mean 

3.82 

3.50 

3.65 

3.33 

3.55 

3.52 

3.55 

Type 

Aircraft 

Process 

Auditing 

Reward 

System 

Quality 

Assurance 

Risk 

Manage 

Command 
&  Control 

Comms/ 
Func  Rel 

Type 

Mean 

VMFA 

3.83 

3.69 

3.77 

3.53 

3.64 

3.62 

3.68 

VMFAT 

3.53 

3.18 

3.36 

3.13 

3.32 

3.17 

3.28 

VMGR 

3.96 

3.74 

3.89 

3.63 

3.92 

3.87 

3.83 

VMA 

3.87 

3.55 

3.65 

3.46 

3.62 

3.58 

3.62 

HMH 

3.81 

3.50 

3.68 

3.28 

3.50 

3.56 

3.55 

HMM 

3.68 

3.38 

3.51 

3.04 

3.37 

3.36 

3.39 

HMLA 

3.84 

3.33 

3.55 

3.16 

3.39 

3.40 

3.45 

HMT 

4.07 

3.53 

3.94 

3.47 

3.85 

3.66 

3.75 

Component 

Mean 

3.83 

3.49 

3.67 

3.34 

3.58 

3.53 

3.55 
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APPENDIX  E.  3RD  MAW  COMPONENT  ONE  LOADINGS 


APPENDIX  F.  SPLUS  CODE  &  OUTPUT  FOR  MCAS  ANOVA  &  MANOVA 

>  tm 

PA  RS  QA  RM  CC  CF 

A  3.75  3.41  3.63  3.12  3.50  3.43 

B  4.07  3.53  3.94  3.47  3.85  3.66 

C  4.00  3.34  3.56  3.30  3.40  3.37 

F  4.02  3.67  3.60  3.65  3.68  3.55 

G  3.79  3.37  3.42  2.98  3.33  3.21 

I  3.98  3.39  3.52  3.28  3.51  3.53 

L  4.02  3.74  3.99  3.56  3.83  3.92 

M  3.61  3.25  3.37  2.99  3.17  3.19 

O  3.58  3.21  3.32  2.82  3.18  3.20 

P  3.74  3.28  3.66  3.20  3.45  3.63 

>  tml_data . frame (Avg  =  unlist (c (tm) ) ,  Squad  =  dimnames(tm) 

111]] [row (tm)].  Comp  =  dimnames (tm) [ [2] ] [col (tm) ] ) 


»  tmaov_anova (aov (Avg  ~  Squad  +  Comp,  data  =  tml)) 
>  tmaov 

Analysis  of  Variance  Table 


Response:  Avg 

Terms  added  sequentially  (first  to  last) 

Df  Sum  of  Sq  Mean  Sq  F  Value  Pr(F) 

Squad  9  2.27  0.252  26.7  3.22e-015 

Comp  5  2.13  0.426  45.1  2.20e-016 

Residuals  45  0.43  0.009 

>  t appl y ( tml [ , "Avg " ] , tml [ , "Comp ” ] , mean ) 

CC  CF  PA  QA  RM  RS 
3.49  3.47  3.86  3.6  3.24  3.42 


>  boxplot (split (tml [, "Avg"], tml [,  "Squad"]  )  ) 
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>  aovl_anova (aov (Avg  -  Squad  +  Comp,  data  =  tml) ) 

>  aov2_aov(Avg  -  Squad  +  Comp,  data  =  tml) 

>  multl_multicomp  (aov2 ,  focus  =  "Comp”,  method=”tukey" ,  plot=T ) 

>  multi 


95  %  simultaneous  confidence  intervals  for  specified 
linear  combinations,  by  the  Tukey  method 

critical  point:  2.976 
response  variable:  Squad 


intervals  excluding  0  are  flagged  by  ****** 


Estimate 

Std. Error 

Lower  Bound 

Upper  Bound 

o 

o 

1 

o 

*TJ 

0.021 

0.0435 

-0.1080 

0.15000 

CC-PA 

-0.366 

0.0435 

-0.4950 

-0.23700 

T*C  ★  ★  'k 

CC-QA 

-0.111 

0.0435 

-0.2400 

0.01840 

CC-RM 

0.253 

0.0435 

0.1240 

0.38200 

*  *  * 

CC-RS 

0.071 

0.0435 

-0.0584 

0.20000 

CF-PA 

-0.387 

0.0435 

-0.5160 

-0.25800 

*  *  *  * 

CF-QA 

-0.132 

0.0435 

-0.2610 

-0.00264 

*  ★  *  * 

CF-RM 

0.232 

0.0435 

0.1030 

0.36100 

*  *  *  * 

CF-RS 

0.050 

0.0435 

-0.0794 

0.17900 

PA-QA 

0.255 

0.0435 

0.1260 

0.38400 

*  *  *  * 

PA-RM 

0.619 

0.0435 

0.4900 

0.74800 

*  *  *  * 

PA-RS 

0.437 

0.0435 

0.3080 

0.56600 

★  ★  *  * 

QA-RM 

0.364 

0.0435 

0.2350 

0.49300 

*  *  *  * 

QA-RS 

0.182 

0.0435 

0.0526 

0.31100 

*  *  ★  * 

RM-RS 

-0.182 

0.0435 

-0.3110 

-0.05260 

★  *  *  * 

-0.6  -0.4  -0.2  0.0  0.2  0.4  0.6  0.8 

simultaneous  95  %  confidence  limits,  Tukey  method 
response  variable:  Squad 
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>  mult2_multicomp (aov2 ,  focus  =  "Squad",  method="tukey" , plot=T) 
WARNING:  Point  out  of  bounds:  x  =  -0.680036,  y  =  11.000000 

>  mult 2 


95  %  simultaneous  confidence  intervals  for  specified 
linear  combinations,  by  the  Tukey  method 

critical  point:  3.3269 
response  variable :  Squad 


intervals  excluding  0  are  flagged  by  '****' 


Estimate 

Std. Error 

Lower  Bound 

Upper  Bound 

A-B 

-0.28000 

0.0561 

-0.4670 

-0.0933 

*  *  *  * 

A-C 

-0.02170 

0.0561 

-0.2080 

0.1650 

A-F 

-0.22200 

0.0561 

-0.4080 

-0.0350 

kkkk 

A-G 

0.12300 

0.0561 

-0.0634 

0.3100 

A- 1 

-0.06170 

0.0561 

-0.2480 

0.1250 

A-L 

-0.37000 

0.0561 

-0.5570 

-0.1830 

A-M 

0.21000 

0.0561 

0.0233 

0.3970 

*  *  *  * 

A-0 

0.25500 

0.0561 

0.0683 

0.4420 

*  *  *  * 

A-P 

-0.02000 

0.0561 

-0.2070 

0.1670 

B-C 

0.25800 

0.0561 

0.0716 

0.4450 

B-F 

0.05830 

0.0561 

-0.1280 

0.2450 

B-G 

0.40300 

0.0561 

0.2170 

0.5900 

*  *  ** 

B-I 

0.21800 

0.0561 

0.0316 

0.4050 

B-L 

-0.09000 

0.0561 

-0.2770 

0.0967 

B-M 

0.49000 

0.0561 

0.3030 

0.6770 

*  *  *  * 

B-0 

0.53500 

0.0561 

0.3480 

0.7220 

*  *  *  * 

B-P 

0.26000 

0.0561 

0.0733 

0.4470 

-k  'k  -k 

C-F 

-0.20000 

0.0561 

-0.3870 

-0.0133 

■k  -k  -k  -k 

C-G 

0.14500 

0.0561 

-0.0417 

0.3320 

C-I 

-0.04000 

0.0561 

-0.2270 

0.1470 

C-L 

-0.34800 

0.0561 

-0.5350 

-0.1620 

**** 

C-M 

0.23200 

0.0561 

0.0450 

0.4180 

k  ~k  k  k 

C-0 

0.27700 

0.0561 

0.0900 

0.4630 

k  k  k  k 

C-P 

0.00167 

0.0561 

-0.1850 

0.1880 

F-G 

0.34500, 

0.0561 

0.1580 

0.5320 

kkkk 

F-I 

0.16000 

0.0561 

-0.0267 

0.3470 

F-L 

-0.14800 

0.0561 

-0.3350 

0.0384 

F-M 

0.43200 

0.0561 

0.2450 

0.6180 

k  k  k  k 

F-0 

0.47700 

0.0561 

0.2900 

0.6630 

kkkk 

F-P 

0.20200 

0.0561 

0.0150 

0.3880 

kkkk 

G-I 

-0.18500 

0.0561 

-0.3720 

0.0017 

G-L 

-0.49300 

0.0561 

-0.6800 

-0.3070 

kkkk 

G-M 

0.08670 

0.0561 

-0.1000 

0.2730 

G-0 

0.13200 

0.0561 

-0.0550 

0.3180 

G-P 

-0.14300 

0.0561 

-0.3300 

0.0434 

I-L 

-0.30800 

0.0561 

-0.4950 

-0.1220 

kkkk 

I-M 

0.27200 

0.0561 

0.0850 

0.4580 

kkkk 

1-0 

0.31700 

0.0561 

0.1300 

0.5030 

kkkk 

I-P 

0.04170 

0.0561 

-0.1450 

0.2280 

L-M 

0.58000 

0.0561 

0.3930 

0.7670 

kkkk 

L-0 

0.62500 

0.0561 

0.4380 

0.8120 

kkkk 

L-P 

0.35000 

0.0561 

0.1630 

0.5370 

kkkk 

M-0 

0.04500 

0.0561 

-0.1420 

0.2320 
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M-P  “0.23000 
O-P  “0.27500 


0.0561 

0.0561 


“0.4170 

-0.4620 


“0.0433  **** 
-0.0883  **** 


M-0 

M-P 

O-P 


-0.6  -0.4  -0.2  0.0  0.2  0.4  0.6  0.8 

simultaneous  95  %  confidence  limits,  Tukey  method 
response  variable:  Squad 


>  aovl 

Analysis  of  Variance  Table 
Response:  Avg 

Terms  added  sequentially  (first  to  last) 

Df  Sum  of  Sq  Mean  Sq  F  Value  Pr(F) 

Squad  9  2.27  0.252  26.7  3.22e~015 

Comp  5  2.13  0.426  45.1  2.20e-016 

Residuals  45  0.43  0.009 

>  aov2 
Call: 

aov( formula  =  Avg  -  Squad  +  Comp,  data  =  tml) 


Terms : 

Sum  of  Squares 
Deg.  of  Freedom 


Squad  Comp  Residuals 
2.27  2.13  0.43 

9  5  45 


Residual  standard  error:  0.0972 
Estimated  effects  are  balanced 


>  summary (aovl ) 
Df 

Sum  of  Sq 

Mean  Sq 

F  Value 

Pr  (F) 

Min. :  5.0 

Min. : 0. 425 

Min. :0. 00945 

Min . : 26 . 7 

Min. : 2 . 22e-016 
1st  Qu . :  7.0 

1st  Qu. : 1 . 280 

1st  Qu . : 0 . 13100 

1st  Qu. : 31 . 3 

1st 

Qu. : 9 . 71e-016 
Median:  9.0 

Median: 2 . 130 

Median : 0 . 25200 

Median: 35 . 9 

Median : 1 . 72e-015 
Mean: 19. 7 

Mean: 1 . 610 

Mean: 0 . 22900 

Mean: 35 . 9 

Mean : 1 . 72e-015 
3rd  Qu. :27.0 

3rd  Qu. : 2 . 200 

3rd  Qu. : 0 . 33900 

3rd  Qu. :40.5 

3rd 

Qu. : 2 . 47e-015 
Max. : 4  5 . 0 

Max . : 2 . 270 

Max . : 0 . 4  2  600 

Max . : 45 . 1 

Max. : 3 . 22e-015 

NA' s : 1 . 00e+000 

NA 1 s :  1.0 
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>  summary ( aov2 ) 


Df  Sum  of  Sq  Mean  Sq 

Squad  9  2.27  0.252 

Comp  5  2.13  0.426 

Residuals  45  0.43  0.009 

>  summary ( anova ( aov2 ) ) 

F  Value  Pr(F) 

26.7  3 . 22e-015 
45.1  2 . 20e-016 

Df 

Pr  (F) 

Sum  ■ 

of  Sq 

Mean  Sq 

F  Value 

Min.:  5.0 
Min.  :  2 . 226-016, 

Min . 

:  0 . 425 

Min. :0. 00945 

Min. : 26 . 7 

1st  Qu. :  7.0 

Qu. : 9 . 7 le-016 

1st  Qu. 

:  1 . 280 

1st  Qu. : 0 . 13100 

1st  Qu. : 31 . 3 

1st 

Median:  9.0 
Median: 1.72e-015 

Median 

:  2 . 130 

Median: 0 .25200 

Median: 35. 9 

Mean: 19.7 
Mean: 1 . 72e-015 

Mean 

:  1 . 610 

Mean: 0 . 22900 

Mean: 35 . 9 

3rd  Qu. : 27 . 0 

Qu. : 2 . 47e-015 

3rd  Qu. 

:  2 . 200 

3rd  Qu . : 0 . 33900 

3rd  Qu. : 40 . 5 

3rd 

Max. : 45 . 0 
Max. : 3 . 22e-015 

NA?  s : 1 . 00e+000 

Max. 

:  2 . 270 

Max. :0. 42600 

Max. : 45 . 1 

NA's:  1.0 
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APPENDIX  G.  3rd  MAW  REGRESSION  DATA 


Data  set  =  maw,  Name  of  Fit  =  LI 
Normal  Regression 
Kernel  mean  function  =  Identity 
Response  =  Incidents 

Terms  =  (Mean  MCAS) 

Coefficient  Estimates 


Label 

Estimate 

Std.  Error  t-value 

Constant 

-94.6783 

171.398  -0.552 

Mean  MCAS 

32.7781 

48.7714  0.672 

R  Squared: 

0. 

0534434 

Sigma  hat: 

29.822 

Number  of  cases: 

10 

Degrees  of 

freedom: 

8 

Summary  Analysis  of 

Variance  Table 

Source 

df 

SS 

MS  F 

p-value 

Regression 

1 

401. 

709 

401.709  0.45 

0.5205 

Residual 

8 

7114 

.82 

889.352 

Lack  of  fit  7 

6993 

.  14 

999.019  8.21 

0.2626 

Pure  Error 

1 

121 

.68 

121.68 

Data  set  =  mawminusF,  Name  of  Fit  =  LI 
Normal  Regression 
Kernel  mean  function  =  Identity 
Response  =  Incidents 


Terms  =  (Mean 

Coefficient  Estimates 
Label  Estimate 

Constant  42.3433 

Mean  MCAS  -8.71304 

R  Squared: 

Sigma  hat: 

Number  of  cases: 
Degrees  of  freedom: 


Std.  Error  t-value 

77.0016  0.550 

22.0366  -0.395 

0.0218454 

12.8019 

9 

7 


Summary  Analysis  o 


Source  df 

Regression  1 

Residual  7 

Lack  of  fit  6 

Pure  Error  1 


Variance  Table 

SS  MS 

25.6212  25.6212 

1147.22  163.889 

1025.54  170.924 

121.68  121.68 


F  p-value 

0.16  0.7043 

1.40  0.5688 
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